Apparatus, system, and method for modulating consolidation of memory during sleep

ABSTRACT

Devices, systems and methods to modify memory and/or cognitive function by delivering a sensory stimulus paired with learned material at opportune physiological periods during sleep. For example, described herein are systems, methods and devices to enhance a user&#39;s cognitive function in such areas as memorization and learning. A machine (e.g., a system or device) may be used to identify opportune periods of the sleep cycle and to deliver a stimulus during specific phases of the sleep cycle to facilitate or interrupt memory consolidation. In some variations the machine records ambient sensory inputs during awake acquisition or reinforcement/relearning and replays all or an extracted form of the ambient sensory stimuli a specified portion of the user&#39;s sleep.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority as a continuation-in-part to U.S.patent application Ser. No. 13/439,715, filed Apr. 4, 2012, and titled“APPARATUS, SYSTEM, AND METHOD FOR MODULATING CONSOLIDATION OF MEMORYDURING SLEEP,” now Publication No. 2012-0251989, which claims priorityto U.S. Provisional Patent Application No. 61/471,526, filed Apr. 4,2011, and titled “APPARATUS, SYSTEM, AND METHOD FOR MODULATINGCONSOLIDATION OF MEMORY DURING SLEEP,” each of which is hereinincorporated by reference in its entirety.

This application also claims priority to U.S. Provisional PatentApplication No. 61/744,826, filed Oct. 3, 2012, and titled “LEARNINGDATABASE AND SCHEDULER SYSTEM AND METHODS FOR MODULATING CONSOLIDATIONOF MEMORY DURING SLEEP,” which is herein incorporated by reference inits entirety.

INCORPORATION BY REFERENCE

All publications and patent applications mentioned in this specificationare herein incorporated by reference to the same extent as if eachindividual publication or patent application was specifically andindividually indicated to be incorporated by reference.

FIELD

Described herein are devices, systems and methods for modulating(including enhancing and/or disrupting) cognition and/or memory. Moreparticularly, described herein are devices, systems and methods forinterfacing computerized platforms to enhance human cognition duringsleep.

BACKGROUND

Proper memory function requires encoding of a memory during learning,consolidation of the memory in the hours and days that follow, andretrieval of the learned content during testing.

Memory consolidation is the process whereby the brain transfers memoriesto long-term storage. Consolidation of memories occurs primarily duringsleep. Deep, or ‘slow-wave’ sleep (SWS), is particularly important forconsolidating long-term memories. Recent advances in the fields ofneurobiology, psychology, and sleep research have characterized theimportant relationship between sleep and memory.

Sleep is required for normal memory consolidation and reduced sleepquality or quantity disrupts memory function. In people, memory andother higher cognitive functions can be improved by increasing sleepquantity or sleep quality. Intensive training or learning causes anincrease in the amount of SWS sleep during a subsequent night,suggesting that this phase of sleep is required for memories to beconsolidated. In rodents, neurobiological studies have shown thatpatterns of activity among neurons in the hippocampus, a key brainregion for memory, occur in a predictable and sequential pattern when arodent is exploring a maze or other environment. The spatial memoryrepresented by this experience is thought to be consolidated duringsleep. Electrophysiological recordings during SWS have been used toidentify ‘replay’ of the patterns of neural activity observed duringprevious experience, suggesting that replay is an important mechanismfor consolidation of memories to long-term storage. Interruption ofreplay during sleep by electrical stimulation disrupts memory formation.

There are several phases of sleep that occur in a repeated cycle. Sleepphases can be identified by differences in brain activity andphysiology, including variation in heart rate, body temperature, andarousal threshold. In humans, sleep is generally described according toa cycle in which rapid eye movement (REM) sleep is followed by non-REMsleep that generally proceeds sequentially through phases S1, S2, S3,and S4. Phases S1 and S2 are generally referred to as light sleep, andphases S3 and S4 are generally referred to as deep or ‘slow-wave’ sleep(SWS).

Normal cognitive function requires sufficient and well-structured sleep.Cognitive impairment due to sleep abnormalities occurs in individualswith neurodevelopmental disorders such as Down syndrome,neurodegenerative disorders such as Alzheimer's disease, various formsof insomnia, sleep apnea, and other pathological conditions. Similarly,reduced memory function unrelated to disease occurs with normal aging,overnight shift work, drug or alcohol use, and other causes of sleepimpairment or sleep disruption. For these various forms of cognitivedysfunction, strategies to alleviate or mitigate cognitive deficits withpharmaceutical, educational, and behavioral interventions have receivedsignificant attention but have not adequately addressed cognitivedeficits. New methods for improving the lives of those with intellectualdisabilities, age-related cognitive decline, and other forms of learningdisability by improving memory and cognitive function are desired.Moreover, healthy, typically-developed students of all ages wouldbenefit from a method for enhancing memory consolidation and thuslong-term memory retention.

The systems, methods and devices described herein may relate toaugmenting or disrupting memory consolidation. These systems, methodsand devices may allow the application of techniques to improve learningand memory non-invasively and without drugs and may engage memoryconsolidation processes that are active during sleep.

Although there is some academic work examining the presentation ofsensory stimulus cues during sleep, this work has, to date, not beenapplied to a home setting in a manner that allows application of thesetechniques by an individual user. For example, in the first publicationto report this effect, contextual presentations of olfactory cues duringa prior learning event, when re-exposed during slow-wave sleep, wereshown to improve the retention of memories formed during the learningevent (Rasch, B., Büchel, C., Gais, S., and Born, J., 2007, Odor cuesduring slow-wave sleep prompt declarative memory consolidation. Science,315, 1426-1429). In the simplest form of this technique, memory may beenhanced by (1) pairing learning with a sound or smell, (2) monitoringsleep during a subsequent night's sleep or nap, (3) detecting slow-wave(deep) sleep, and (4) re-presenting the sensory cue. Published studiesof controlled, clinical studies of this technique have reported up toabout 30% improvements in memory in healthy young adults (Diekelmann,S., Büchel, C., Born, J., and Rasch, B., 2011, Labile or stable:opposing consequences for memory when reactivated during waking andsleep. Nature neuroscience). Unfortunately, these studies provide littleguidance on the application of these results in a home or user-appliedsetting, outside of a controlled laboratory setting.

Further, additional research has shown that misapplication of thesetechniques may lead instead to a decrease in memory. In contrast to thestudies that reported enhanced memory after sensory re-presentationduring sleep, re-presenting the sensory stimulus from training to thesubject during wakefulness leads to a reduction in memory performance(Diekelmann et al., 2011). This finding also suggests methods forabolishing undesired memories such as those associated with traumaticevents that lead to post-traumatic stress disorder (PTSD). Memoryre-consolidation occurs when a memory that has been successfully storedin long-term memory is recalled. Maintenance of the memory in long-termmemory after this event of memory recall requires active neurobiologicalprocesses to re-consolidate the stored memory trace. Accordingly,methods that selectively disrupt memories that are maladaptive, relatedto psychiatric conditions, or otherwise unwanted would be of greatbenefit to many.

Memory consolidation can also be reduced by disrupting sleep or bydepriving a subject of sleep. Studies in humans and animals have shownthat sleep deprivation or disruption of sleep after a training eventlead to reduced memory performance.

Systems and methods for modulating memory consolidation during sleephave been previously described, but these disclosures do not describehow to select, prioritize, and schedule contextual sensory stimuli to bepresented to a subject in order to achieve effective modulation ofmemory consolidation for multiple learning and sleep consolidationevents across one or more days and nights. The systems and methods forconfiguring, populating, and querying a learning database of trainingcontent and contextual sensory stimuli described here are useful forselecting, prioritizing, and scheduling memory consolidation events.

SUMMARY OF THE DISCLOSURE

Described herein are devices, systems and methods to deliver a sensorystimulus or stimuli at opportune physiological periods to enhance theuser's cognitive function in such areas as memorization and learning. Amachine (e.g., a system or device) may be used to identify opportuneperiods of the sleep cycle and to deliver a stimulus during specificphases of the sleep cycle, for example to facilitate or interrupt memoryconsolidation. In other embodiments, the machine records ambient sensoryinputs such as those detected by auditory, somatosensory, olfactory,gustatory, visual, vestibular, or sensory systems during a period oflearning and stores these stimuli in a database for subsequentre-presentation during sleep.

The devices and systems may determine the phase of sleep for a user aspart of an integrated device or system that also determines the sensorystimulus and presents the sensory stimulus during training and/or sleep.The determination of sleep phase by monitoring an individual duringsleep may be referred to as sleep staging. Sleep staging may beperformed using the traditional Rechtschaffen & Kales rules, whichclassify sleep into six separate stages: wake, rapid eye movement (REM)sleep, S1 (light sleep), S2 (light sleep), S3 (deep sleep), and S4 (deepsleep). Alternative systems for sleep staging have been described andare known to those skilled in the art. Techniques for monitoringphysiological changes associated with different stages of sleep mayinclude electroencephalography (EEG) recordings of brain activity,electrooculagraphy (EOG) of eye movement and ocular muscle contractions,electrocardiography (ECG) of heart beats, as well as heart rate or heartrate entropy, respiratory rate, body temperature, eye or body movements(actigraphy), and other techniques. A variety of devices and sensors canbe used to monitor physiological changes associated with sleep phasesand may be incorporated into the devices, systems and methods describedherein.

In addition to sensing the phase of sleep, the devices and systems mayalso deliver a stimulus, such as an electrical or sensory stimulus, forthe purpose of modulating the phase of a user's sleep or modulating thequality of sleep and/or the quality or intensity of brain rhythms duringa particular phase of sleep. In particular, the stimulus may mimic orrepeat a stimulus that was intentionally or inadvertently paired withthe “learning” of the memory to be modulated.

Learning of any appropriate material may be modulated (e.g., enhanced,inhibited) by the systems and methods described herein. For example, insome variations, the devices and systems described herein enhancelearning of material to be compiled for a user from several sources thatmay include: 1) material entered into a computing device by the user ora third party that may be stored locally on the device or stored on aremote server or intermediate device; 2) material that may be in avariety of electronic formats that can be uploaded by the user or thirdparty to a computing device for storage through the Internet on a remoteserver or on an intermediate device; 3) material chosen by the user froma pre-determined set of training material that may be provided by athird party or the device; 4) material that may be generated based onthe user's location at a particular time or ‘check-in’ by a servicehaving functionality similar to that of Gowalla or Foursquare; 5)material generated based on the user's interests as indicated by theuser, by a third party, or by data mining; 6) material that may besupplied by a third party such as a teacher, work colleague, friend,advertiser, or other individual or entity; or 7) material supplied byother suitable means appreciated by one skilled in the art, that may becurrently known or hereafter developed.

In some variations, the devices and systems described herein may benetworked such that user information and device control can be managedthrough interactions with any combination of the devices. Exemplaryinteractions include: 1) the ability for the user to selectphysiological recording settings from a personal computer or othermachine, which further synchronizes information with a base station, 2)the ability for sleep quality information acquired from a base stationto be stored and analyzed on a personal computer, and/or 3) the abilityfor remotely communicating with the base station from a learningenvironment.

In one embodiment, the system or device has the ability to be networkedwith remote servers, other nearby components of the system or device,and third party servers and devices. The networking of devices offersadditional functionality, ease of use, physiological sensing, or datamanagement as each networked or otherwise connected device may havedifferent advantages in different phases of the training program. Forexample, networking of devices may permit data on learned material, pastsleep stimulus exposures, awake sensory exposures, cognitiveperformance, or event attendance at a particular location, or event,such as in a lecture hall, to be used to deduce probable stimuliexposures or training content of interest.

In general, the devices described herein may be controlled by the user.Thus, any of the variations described herein may be configured so that auser may activate the device (e.g., trigger the learning/trainingsession and later sleep consolidation sessions) without the need foradditional intervention (e.g., from a technician, or the like). Forexample, a user may control the device through the machine's userinterface; said interface may exist on the device, through the Interneton a remote server, through a web-based software application, or throughan intermediate device. An example interaction between the user and thedevice through the user interface may be in selecting which previouslearned items the user wishes to further practice, or in defining theimportance of a single or group of learned item(s).

In some variations, a user or third party may receive feedbackconcerning various aspects of the device's function, including but notlimited to the quality of sleep, learning content presented, sensorycues presented, and memory function. This feedback may be generated inreal-time or at a delay. The feedback may be delivered to the user or athird party by various routes including, but not limited to sensoryfeedback such as by visual, auditory, or haptic stimuli or informationdisplayed on a computer screen, handheld device, or other electronicdevice.

In general, the stimulus applied during learning and again during sleepconsolidation may be any appropriate sensory stimulus. In particular,stimuli that are below the threshold for waking the user (e.g.,non-distracting sensory stimuli) are used. The mode of the stimuli mayinclude one or more of: audible, tactile (including somatosensory),olfactory, vestibular, gustatory, visual, or the like. In somevariations, the system or device includes a number of predeterminedsensory stimuli, and may choose from this pool of sensory stimuli topair the particular, specific sensory stimuli with a particular trainingsession and/or with a particular piece of information to be learned. Insome variations, the sensory stimuli may be multi-modal.

In some variations, the system or devices may be configured to recordand provide sensory stimuli that are copied or extracted from ambient orenvironmental sensory information not controlled by the invention. Forexample, the system or devices described herein may include an ambientrecorder (“sensory recorder”) for recording or extracting backgroundsensory stimuli from during a training period. In some variations thesystem may receive information on sensory stimulus provided by audioplayers, computers, television/video players, and the like. In somevariations the system or device may communicate electronically with suchperipherals to determine what sensory stimulus was being presented bythese peripherals during the learning phase.

The device and systems described herein may interface with a networkeddevice to enable selection of sensory information from a database. Thecontent of such a database could be populated from user-uploaded contentor from third parties through a web-browser or similarly effectivewidget. Third parties may include educational entities andsocially-derived networks.

In some variations, the systems or devices may also include the abilityto deliver a reconstruction of recorded ambient stimuli, or acomplimentary percept, such as one that has undergone signal processingto identify signals of interest or high variance, during targeted phasesof the users' sleep cycle.

In general, the devices and systems described herein may be configuredfor use with one or more user. For example, a system or device may keeptrack of which users receive which stimuli at which time and sleepphase. As mentioned, the system and device may be configured to ensurethat a particular stimulus is paired with a particular training sessionand/or material to be learned for a particular user. Thus, a device maybe configured to determine user identity to help insure that the sameuser receives novel stimuli for new training sessions/new material.Thus, in some variations the system may use biometric data todetermine/confirm user identity. In some variations the systems and/ordevices may request user name and/or password/identity codes. Examplesof biometric data may include fingerprint, or other biometricidentifiers. In some variations the system is configured to assume thatit will only be used for a single user. The pool of sensory stimuli maybe finite, and in some variations the device or system may indicate whenthe maximum number of sensory stimuli has been used. In some variations,the system or device may be configured so that a sensory stimuli modulemay be removed/added/replaced to provide new or additional sensorystimuli for pairing with training sessions or learned material.

In some embodiments, learning material is presented by the machine or bysoftware on a computer, phone, or other electronic device (currentlyknown or hereafter developed) at the same time as sensory stimuli or inclose temporal relation to the sensory stimuli.

Physiological changes that occur in response to presentations of sensorystimuli may be used for feedback modulation of the delivered stimuli.These physiological changes may be monitored during sleep, duringwakefulness, or during training or testing. The variety of physiologicalfeatures that can be monitored will be appreciated by one skilled in theart and may include brain rhythms that relate to attention, memoryprocessing, or other cognitive function; heart rate, heart beat entropy,or ECG signals; respiratory rate or entropy; pulse oximetry (bloodoxygen content, SpO₂); galvanic skin responses (GSRs); arousal levels;eye gaze; posture; muscle tone; or other physiological signals ofinterest or appreciated by one skilled in the art.

Desirable feedback in response to identified physiological features mayinclude, but are not limited to, modifying the intensity, modality, orother aspects of sensory stimulus presentation; modifying the rate,content, or difficulty of training content; delivering a reminder cue tomodify attention, gaze, or other aspects of cognition or physiology; ordelivering stimuli to affect brain rhythms and/or cognitive processes,including but not limited to increasing the frequency, intensity, orspatial extent of slow wave (delta) rhythms. Moreover, stimuli may bemodulated to recapitulate prior physiological activity measured duringpast successful sleep sessions. For example, a strong stimulus intensitycould negatively affect the stability of the current sleep phase,whereas a weak stimulus intensity may not be sufficiently salient towarrant memory consolidation. By monitoring the effect on sleep, memory,or other aspects of physiological function, the present invention canchoose the appropriate stimulus parameters for a particular user given aparticular set of physiological measurements.

Third parties may be granted access to control the device as well asgather information from the device through the Internet on a remoteserver, or through an intermediate device. A third party may providespecific instructions as to how the device is to interact with the user.In some variations a third party may receive information such asdurations of sleep states or assessments of memory performance.

Third parties may be granted access to submit data to components of thedevice to improve the ability of the device to aggregate learningcontent and to integrate the device's capabilities with the thirdparty's objectives, as well as gather information from the devicethrough the Internet on a remote server, through a web-based softwareapplication, or through any suitable intermediate device currently knownor hereafter developed.

One variation of a device or system is configured to aid or enhance thedevelopment of a skill. For example, in some variations, the system maybe configured to aid in learning foreign languages or technical softwarelanguages. Training sessions may be paired with a sensory stimulus. Ingeneral, the sensory stimulus is distinct from the material beingtrained. For example, the sensory stimulus may be a primitive (e.g., ascent, tone, touch, or the like) that does not, in the absence of beingpaired with the training session, evoke the trained material.

In general, the systems or devices described herein may be used to trainvirtually any subject matter. For example, the systems or devices and/orrelated methods may be used for learning vocational trade skills, suchas, for example, training of automotive mechanics. In one variation, thesystems or devices may be used for test preparation for standardizedtests such as those required for admission to primary or secondaryschools, universities, post-graduate programs, or other academicpursuits; tests required for admittance to a specific professional groupincluding but not limited to the state bar exam for lawyers, theCertified Public Accountant (CPA) exam for accountants, the medicalboard exams for doctors, the Series 7 exams for brokers, or otherprofessional tests or exams that would be known to one skilled in theart of test preparation. In some variations, the systems and devicesdescribed herein may be used for experiential learning such as occursduring residency training for medical doctors, strategy learning forprofessional athletes (such as learning a playbook for footballplayers), or other applications appreciated by one skilled in the artfor which learning by observing and learning by doing are fundamental.Learning involving rote memorization may also be enhanced using thedevices, systems and methods described herein.

Thus, the systems, devices and methods described herein may beparticularly useful for repeated learning using these techniques. Thesystem, devices and methods described herein are particularly wellsuited for repeated use where it is desirable or necessary to enhancelearning of more than one piece of information or task skill. Becauserepeated use requires the use of multiple stimulations, repeated at oneor more times following a learning session, a device useful for repeatedlearning is typically capable of distinguishing between learning trialsand automatically tracking stimuli and learning sessions.

In one variation, the system and device may be used to learn more aboutanother person's interests or experiences by sharing content learned orstimuli delivered by the device. This embodiment could be of use to helpprepare for an interview, a sales meeting, or a blind date. Similarly,in some variations, the device, system and/or related methods may beused in military, law enforcement, or intelligence gatheringapplications, including those related to interrogation. In anotherembodiment, the invention may be used for training of soldiers, membersof law enforcement, or intelligence agents.

The devices and systems described herein may also be used as part of atherapeutic method to treat a patient. For example, the devices, systemsand methods may be used to improve memory and/or cognitive function byindividuals with inherited neurodevelopmental disorders characterized bylearning, memory and/or cognitive deficits including Down syndrome, Rettsyndrome, fragile X syndrome, neurofibromatosis type 1, tuberoussclerosis, phenylketonuria, maple syrup urine disease, and otherinherited neurodevelopmental disorders appreciated by one skilled in theart, as well as disorders such as autism spectrum disorders which aregenerally diagnosed in the first five years of life and may be due togenetic and/or environmental causes.

An embodiment that applies the device and/or related methods may be usedto improve memory and/or cognitive function by individuals withcognitive and/or memory deficits associated with normal aging orneurodegenerative disorders including Alzheimer's disease, Parkinson'sdisease, frontotemporal dementia, and other age-related orneurodegenerative disorders appreciated by one skilled in the art.

In some variations, the systems, devices and/or related methods may beused by individuals with disorders of sleep such as central sleep apnea,obstructive sleep apnea, insomnia, and other forms of sleepabnormalities appreciated by one skilled in the art, including but notlimited to those that lead to reduced memory function or cognitiveimpairment.

In some variations, the devices, systems and/or methods may be used as asubstitute or supplement to drug therapy to enhance or disrupt memoryformation or eliminate or otherwise modify existing memories.

In some variations, the systems, devices and/or related methods may beused by individuals with disorders for which memory disruption isdesired such as post-traumatic stress disorder (PTSD), obsessivecompulsive disorder, depression, or other disorders appreciated by oneskilled in the art.

These system and devices may be used by adults and/or children. Forexample, the systems, devices and/or related methods described hereinmay be adapted for use by babies, toddlers, or pre-kindergarten-agedchildren. In this application, one embodiment is a device built into atoy that a child may interact with and/or a piece of clothing intendedfor the child to wear to bed.

In some variations, the systems, devices and/or related methods may beused prenatally. The sleep state of a fetus may be determined bymonitoring movement, heart rate, respiratory rate, brain rhythms, orother physiological correlates of sleep in the womb with detectorsplaced on or near the mother. The fetus may be presented with auditorytraining material (e.g. words and definitions or other language content)combined with a neutral sound or haptic stimulus and the neutral stimuliare re-presented when the fetus is sleeping or, in some embodiments,more specifically in deep sleep.

In some variations the systems or devices may use training optimizationalgorithms to determine what content is presented at what interval toreduce the amount of time required for training.

The sensory stimulus may be optimized. In some variations, the sensorystimulus provided may be optimized based on the information to belearned. As mentioned, the sensory stimulus provided may benon-interruptive, and may be configured to be innocuous so as not tointerrupt the concentration of the user during the training sessionand/or not to awaken or disrupt the user's sleep during sleepconsolidation. In some variations the sensory stimulus is provided bythird parties (e.g. advertisers) who pay to have a user train with aparticular scent, jingle, or other stimulus during training with thedevice. In another embodiment, a user could purchase a single stimulusor set of stimuli similar to how one buys a ringtone or rights to acopyright-controlled stock image.

An embodiment could populate a user's training set through the Interneton a remote server, or through an intermediate device according to acode (e.g. quick response (QR) code, hashtag, hyperlink, etc.). Onespecific embodiment would be a QR code at a museum exhibit so thatcontent about the exhibit could be learned at another time. Anotherspecific embodiment would be a QR code on an informational plaque at apark or monument so that content about the geography, ecology, orhistory of that location could be learned at another time.

An embodiment could select learning material based on location data,social data (e.g. who are you with at a particular time—are they adevice user, too?), and/or timing data (e.g. what time did you enter aparticular classroom, thus defining whether your training contentdatabase should be populated with content about physics (your class) orchemistry (the preceding class)).

In some variations the system or device may select learning material,sensory stimuli, or other aspects based on a user's use of a socialnetwork, such as one that includes the functionality of Facebook,Twitter, or Myspace.

In general, the sensory stimulus is unique to a particular learningsession or subject matter. Thus, in variations the linking of thesensory stimulus to the subject matter may allow association acrosstraining sessions. For example, in some variations the system (e.g.,using control or system logic) may select stimuli for the purpose ofinvoking transitive inference between content to be learned. In a simpleexample, a user desires to associate the word “orange” with the pictureof an orange. While being presented with an olfactory cue, the user isalso presented the word “orange.” In another learning event, the sameolfactory cue is co-presented with the picture of an orange. The usersubsequently uses transitive inference to associate the word with thepicture, without having a simultaneous presentation of the two items.

In some variations, the system or device may use analytical and/or datamining techniques to determine interests, experiences, and/or previouslylearned content in order to select a stimulus or set of stimuli thatengage transitive inference processes between previous experience andnew learning content. These techniques may be provided as logic (e.g.,stimulus selection logic) configured as hardware, software, firmware, orthe like.

In some variations an automated algorithm (relationship logic)determines which content that has been added to a user's contentdatabase is related, similar or coupled (e.g. all the state capitals,the word ‘sleeping’ in different languages, or a sequence of stepsrequired to perform CPR), then applies the same sensory stimulus (e.g.the sound of snoring or the scent of a rose) for these pieces oftraining content, even if the learning events are separated in time byminutes, hours, days, weeks, months, or years. This embodiment may beconsidered as related to the concept of transitive inference. In arelated embodiment, a curated service, third party, or socially-derivednetwork may determine whether content to be learned is related, similaror coupled and may use this determination to determine whether the sameor similar sensory stimulus should be presented for these items ofcontent to be learned. In another embodiment, the determination ofwhether training content is related for purposes of choosing identicalor similar sensory stimuli to present during training may be made basedon whether other users have coupled such content. For instance, if otherusers had experienced memory improvements due to transitive inference byusing a similar stimulus to associate with content to be learned.

In general, the systems, devices and methods described herein monitorthe user to automatically determine the sleep phase for the patient, andtrigger replay of the sensory stimulus upon or after detection of aparticular sleep phase. The sleep phase may be a typical sleep stage(e.g., slow-wave sleep) or a variation of a typical sleep stage. In somevariations, the user may have their sleep modified when using thedevice. For example, a user may take a drug or drugs to increase thefrequency, changes the lengths of bouts of, or otherwise enhance thequality or quantity of slow-wave sleep to improve memory consolidation.Such drugs could include gamma-hydroxybutyric acid (GHB, also referredto as sodium oxybate), adenosine A1 receptor agonists, 5-HT2 antagonistsor other drugs known to one skilled in the art that increase slow-wavesleep quantity, quality, or intensity. In some variations, the subjectmay take a drug or drugs that disrupts or reduces slow-wave sleep tointerfere with normal memory consolidation. Such drugs could includehypnotics such as flurazepam, benzodiazepine antagonists such asflumazenil, or other drugs known to one skilled in the art that decreaseslow-wave sleep quantity, quality, or intensity. In some variations theuser may take a drug or drugs to change the frequency, lengths of boutsof, or otherwise modulate the quality or quantity of a particular phaseof sleep that may include light sleep, deep sleep, REM sleep, stages S1,S2, S3, or S4 of non-REM sleep, or other identifiable phases of sleep.

Also considered herein is the use of the devices or systems describedherein to use one or more techniques (e.g., electrical stimulation,sensory stimulation or other methods) to induce brain rhythms at deltafrequencies to modulate the functional properties of slow-wave sleep toimprove memory consolidation processes. Similarly, systems, devices andmethods may use electrical stimulation, sensory stimulation or othermethods to disrupt brain rhythms at delta frequencies to modulate thefunctional properties of slow-wave sleep to interfere with memoryconsolidation processes. Electrical stimulation, sensory stimulation orother techniques may be used to induce brain rhythms at otherfrequencies or with other spatial temporal patterns in order to affectbrain rhythms and underlying cognitive processes.

Also described herein are devices, system and methods that use feedbackfrom recording or monitoring of physiological or other parameters thatcorrespond to sleep state or arousal level to define the intensity,modality, and/or specific stimulus delivered during sleep.

For example, described herein are systems for improving memory. Thesesystems may have a training mode and a sleep consolidation mode, and mayinclude: a user interface comprising a control allowing a user to switchthe device to the training mode to indicate a training session; asensory stimulator configured to provide a plurality of distinct sensorystimuli; a sleep monitor configured to monitor a user's sleep state; anda controller comprising control logic receiving input from the userinterface and configured to select a distinct sensory stimulus for aspecific training session and to control the application of the distinctstimulus during the specific training session, and further wherein thecontroller receives information on the user's sleep state from the sleepmonitor, and controls the sensory stimulator to apply the distinctsensory stimulus from the specific training session when the user isexperiencing a specified sleep stage during a sleep consolidation modefollowing the specific training session.

The sensory stimulator may be configured to provide a plurality ofnon-distracting sensory stimuli. The sensory stimuli may be, forexample, olfactory, auditory, or tactile. In some variations the sensorystimuli is predetermined (e.g., the sensory stimulator or controller mayinclude a set of sensory stimuli for delivery by the sensorystimulator); in some variations the sensory stimulator is configured toprovide an ambient sensory stimulus recorded during the trainingsession. For example, the sensory stimulator may comprise an ambientrecorder for recording the ambient stimulus wherein the ambient recorderis configured to record one or more of ambient sounds, ambient odors,and ambient sensations.

Any of the systems or methods described herein may be configured so thatthe specified sleep stage is predetermined from the known sleep stages(e.g., slow wave sleep, light sleep, REM sleep, phases S1, S2, S3, or S4of non-REM sleep, etc.) or combinations of sleep stages. For example, insome variations the sleep stage is slow wave sleep. In some variations(e.g., for reconsolidation of amydalar memories that may be importantfor PTSD) the sleep stage is rapid eye movement sleep.

In any of the variations described herein the systems and devices mayinclude a memory (e.g., a computer or digitally readable/writablememory). This memory may be connected directly or remotely to thecontroller. The memory may be configured to store information thatindicates one or more of which sensory stimuli have been applied forspecific training sessions, the sleep state of user, and completion ofapplication of a sensory stimulus during a sleep consolidation modefollowing a specific training session. In some variation the memory isimportant for storing and providing user configuration. For example, thedevice or system may read a user configuration file or memory todetermine what sensory stimuli have been used, or are available for use,and/or for determining what training has occurred, or has been pairedwith a sensory stimuli. The configuration file may also store userinformation (e.g., biometric information) and/or access information. Thecontroller may be configured to read information from the memory.

In general, the user interface is adapted so that the user may readilyand easily control operation of the system or device. For example, theuser interface may include at least one of: a switch, a toggle, abutton, a slider, a knob, or a touchscreen, and may indicate (viainstructions, menus, or the like) what options the user may select. Theuser interface may provide visual, audible, or tactile feedback on thestatus or operation of the device and/or system.

Any appropriate sleep monitor may be used. The system or device mayinclude sleep monitoring logic to determine (based typically oninformation provided by a sleep monitor) what sleep state the user isin. This determination may be probabilistic (e.g., the logic mayindicate a user is in a particular sleep state, or is not even sleeping,when user indicators (e.g., movement indicators, thermal indicators,electrical indicators, etc.) indicate that the likelihood of aparticular sleep state is above some threshold). In some variations thesleep monitor comprises a non-contact sleep monitor. For example, thesleep monitor may be positioned near the sleeping user and may indicate(based on motion) an approximation of which sleep state the user is in.

In any of the variations described herein the system or device may beconfigured to reside all or partially in a housing. For example, thehousing may at least partially enclose the user interface, sensorystimulator, and controller, and/or other components. In some variationsthe system or device is portable. For example, a portable system mayinclude a housing and may be battery-powered or wall (plug-in) powered.The devices or systems may be lightweight, particular the portablesystems. For example the device or system may weigh less than 20 pounds,less than 15 pounds, less than 10 pounds, or less than five pounds. Insome ultra lightweight configurations the system weighs less than apound. For example, a system or device may be configured to operate on ahandheld device (e.g., iPhone™, iPad™, Android, or other mobile devicecapable or running application software). For example, the control logiccomprises an application configured to be executed on a mobile device.

The system or device may include a handle. In general, these systems anddevice are intended for a user to operate without requiring additionalassistance, at home (e.g., for personal use).

Any of the devices or systems described herein may include acommunications module coupled to the controller configured to allowcommunication with a remote site.

Thus, for example, also described herein are portable user-controllabledevices for improving memory, the device comprising: a user interfacecomprising a control allowing a user to place the device into a trainingmode indicating a training session; a sensory stimulator configured topresent a plurality of distinct sensory stimuli; sleep monitoring logicconfigured to determine when the user is in a specified sleep state; anda controller comprising control logic configured to determine a specificsensory stimulus received by the user concurrent with a particulartraining session; wherein the controller is further configured toreapply the specific sensory stimulus when the user is in a specifiedsleep state following the training session. As mentioned, the portablesystem or device may include a housing at least partially enclosing theuser interface, sensory stimulator, and controller. The portable systemor device may also include a sleep monitor configured to monitor theuser's sleep state.

Also described herein are systems for improving memory having a trainingmode and a sleep consolidation mode, the system comprising: a userinterface comprising a control allowing a user to switch the device tothe training mode to indicate a training session; a sensory stimulatorconfigured to play back an ambient stimulus recorded during a trainingsession; a sleep monitor configured to monitor the user's sleep state;and a controller comprising control logic receiving input from the userinterface and configured to cause the system to record the ambientstimulus during the training session, further wherein the controllerreceives information on the patient's sleep state from the sleepmonitor, and controls the sensory stimulator to apply the recordedambient stimulus when the user is experiencing a specified sleep stagefollowing the specific training session.

As mentioned above, any of the systems or devices described herein maybe configured to use ambient sensory stimuli, including ambient noise.In some variations the system or device may therefore include an ambientrecorder for recording the ambient stimulus. The ambient recorder may bepart of the sensory stimulator, or it may be a separate element. Forexample, an ambient recorder may be configured to record one or more ofambient sounds, ambient odors, and ambient sensations. In somevariations the sensory stimulator is configured to access one or moresources of ambient stimuli that are active during the training session.The one or more sources of ambient stimuli may include one or more of:audio players, computers, televisions, mobile devices, scent releasingdevices, and massage/vibratory devices, or any other device configuredto deliver a sensory stimulus to the user.

Also described herein are methods of improving memory with auser-controlled device. For example, the method may include the stepsof: selecting, in a user-controlled device, a specific sensory stimulusthat is received by the user during a first learning period; detecting,with the user-controlled device, a specified sleep stage in the userfollowing the first learning period; and delivering, from theuser-controlled device, the specific sensory stimulus to the user duringthe specified sleep stage following the first learning period. In somevariations, the method further includes delivering, from theuser-controlled device, the specific sensory stimulus to the user duringthe first learning period. In some variations the method includes usingambient sensory stimuli. For example, the method may include selectingthe specific sensory stimulus by recording an ambient sensory stimulusduring the first learning period.

As mentioned above, delivering the specific sensory stimulus may includedelivering a non-distracting sensory stimulus.

In some variations, the method includes the steps of: selecting, in theuser-controlled device, a second sensory stimulus that is different fromthe first sensory stimulus and that is received by the user during asecond learning period; and delivering the second sensory stimulus tothe user during a specified sleep stage following the second learningperiod.

The method may also include the step of switching the user-controlleddevice so that the device is in a training mode during the firstlearning period and in a sleep consolidation mode during delivery of thesensory stimulus to the sleeping user.

Selecting the sensory stimulus may include choosing a sensory stimulusfrom among a plurality of non-distracting sensory stimuli that have notpreviously been delivered by the user-controlled device during alearning period. In some variations, selecting the sensory stimuluscomprises selecting one or more of: an olfactory stimulus; an auditorystimulus; and a tactile stimulus.

Detecting a specified sleep stage in the user following a learningperiod (e.g., the first or a second learning period) may includemonitoring the users sleep state without contacting the users head orportion of the user's body.

In some variations, the method may also include storing in a memoryinformation that indicates one or more of: which sensory stimulus hasbeen selected, a sleep state of user, and completed delivery of asensory stimulus during the specified sleep stage.

Also described herein are methods of disrupting memory formation in auser, the method comprising: delivering, from a user-controlled device,a specific sensory stimulus to the user during a training periodcomprising recollection or re-experience of information to be forgotten;detecting, from the user-controlled device, a specified sleep stage inthe user; delivering, from the user-controlled device, the specificsensory stimulus during the specified sleep stage; and disrupting thespecified sleep stage.

As mentioned above, disruption of memory may be particularly valuable intreating a disorder of memory such as post-traumatic stress disorder(PTSD) and the like. The step of disrupting the specified sleep stagecomprises waking the user and/or modulating brain rhythms active duringthe specified sleep stage, and/or shifting the user to a different sleepstage and/or disrupting the sleep stage through brain stimulation. Asmentioned above, the specified sleep stage may be any appropriate sleepstage, including slow wave sleep or rapid eye movement sleep.

Also described herein are methods of treating a user for a disorder ofmemory with a user-controlled device, the method comprising: delivering,from the user-controlled device, a specific sensory stimulus to a userhaving a disorder of memory during a first learning period; detecting,with the user-controlled device, a specified sleep stage in the userfollowing the first learning period; and delivering, from theuser-controlled device, the sensory stimulus to the user during thespecified sleep stage following the first learning period.

Any appropriate disorder may be treated. For example, the disorder maybe a disorder of memory that is a neurodevelopmental disorder, such asone or more of: Down syndrome, Rett syndrome, fragile X syndrome,neurofibromatosis type 1, tuberous sclerosis, phenylketonuria, maplesyrup urine disease, autism spectrum disorders. In some variations, thedisorder of memory is a cognitive and/or memory disorder. In somevariations, the disorder is a neurodegenerative disorder, such as one ormore of: Alzheimer's disease, Parkinson's disease, Huntington's disease,frontotemporal dementia. In some variations, the disorder is one or moreof a sleep disorder, central sleep apnea, obstructive sleep apnea, andinsomnia.

Also described herein are systems and methods useful for achievingeffective modulation of memory consolidation during sleep. Moreparticularly, described herein are devices, systems, and methods forachieving effective modulation of cognition or memory over one or moredays and/or nights, including devices, systems, and methods forconfiguring, populating, and querying a learning database with trainingcontent and contextual sensory stimuli co-presented with the trainingcontent, as well as devices, systems, and methods for selecting,prioritizing, and scheduling contextual sensory stimuli to be presentedduring study and sleep. Systems are described that are configured todetermine how to use training content from the learning database todetermine a training schedule for repetition of sensory stimuliassociated with particular training content over one or multiple nightsof sleep.

As described in greater detail below, any of the systems, devices andmethods described herein may be adapted specifically for use withelectronic media. Use with electronic media has particular advantages.In particular, the systems, methods and devices may be configured sothat the electronic media is analyzed and features (e.g., content-basedfeatures and/or contextual features, including structural features) ofthe electronic media may be used to determine/select the sensory stimuliand/or the manner in which the sensor stimuli is delivered or replayed.

For example, described herein are systems for improving learning,memory, or learning and memory from an electronic media. Such systemsmay include a training mode and a sleep consolidation mode, and furthercomprise: a sleep monitor configured to monitor a user's sleep state; asensory stimulator configured to provide a plurality of distinct sensorystimuli based on the electronic media; a controller comprising controllogic configured to select a sensory stimulus for a specific trainingsession based on the content of the electronic media, and to control theapplication of the sensory stimulus, wherein the controller receivesinformation on the user's sleep state from the sleep monitor, andcontrols the sensory stimulator to apply the contextual sensory stimulusfrom the specific training session according to a training schedule whenthe user is experiencing a specified sleep stage during a sleepconsolidation mode; and a learning database configured to be populatedwith training content and sensory stimuli co-presented with the trainingcontent, wherein the controller is configured to determine which sensorystimuli to use based on the electronic media.

As mentioned above, the sensory stimulator may be configured to providean ambient sensory stimulus recorded during the training session. Thecontroller may also be configured to determine which sensory stimuli touse based on the content of the electronic media. In general, thecontent of the electronic media may refer to the subject matter content(e.g., what is being delivered), the content delivery (e.g., language,modality, appearance, etc. of the delivered content), the relation ofthe content to previously delivered content, etc.

In some variations, the controller is configured to determine how toapply the sensory stimuli based on the content of the electronic media.For example, the number of sensory stimuli to deliver, the intensity ofthe delivered sensory stimuli (e.g., volume of the sensory stimuli forauditory stimuli, etc.), or the like.

In some variations the controller is configured to determine whichsensory stimuli to use based on the structure of the electronic media.In general, the structure of the electronic media may refer to thedelivery characteristics of the electronic media, such as the length(e.g., duration) of the delivered electronic media, the type ofelectronic media, etc.).

The system may also include a memory configured to store informationthat indicates one or more of: which sensory stimuli have been appliedfor specific training sessions, the sleep state of user, and completionof application of a sensory stimulus during a sleep consolidation modefollowing a specific training session.

Any appropriate electronic media may be used. For example, theelectronic media may comprise an electronic book (e.g., e-book), adigital recording, etc. Any of the device/system variations describedherein may include an analysis module configured to pre-analyze theelectronic media to determine sensory stimuli to apply. An analysismodule may include software, hardware, and/or firmware that isconfigured to review and analyze the electronic media before and/orduring delivery. The analysis module may determine characteristics,including content-based characteristics, such as subject mattercharacteristics or the like, and context-based characteristics (e.g.,structural characteristics, such as the length of the material to bedelivered, etc.). The analysis module may include analysis logic forrecognizing content or context characteristics. In some variations theanalysis module includes analysis logic configured as a neural networkfor recognizing content and/or context. In some variations the analysismodule is configured to read one or more “tags” associated with theelectronic media that determine the content and/or contextcharacteristics.

Also described herein are portable user-controllable device forimproving learning and memory from an electronic media, the devicecomprising: a user interface comprising a control allowing a user toplace the device into a training mode before the user experiences theelectronic media; a sensory stimulator configured to present a pluralityof distinct sensory stimuli concurrently with a user's experience of theelectronic media; sleep monitoring logic configured to determine whenthe user is in a specified sleep state; and a controller comprisingcontrol logic configured to determine a sensory stimulus based on theelectronic media for a particular training session; wherein thecontroller is further configured to reapply the specific sensorystimulus when the user is in the specified sleep state following thetraining session; wherein the controller is configured to determinewhich sensory stimuli to use based on the content, structure or contentand structure of the electronic media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a schematic view of one variation of a system/device forimproving memory during sleep as described herein.

FIG. 1B illustrates functional elements of another variation of a systemas described herein.

FIG. 2 shows a timeline illustrating how the user may interact with asystem in an exemplary usage in accordance with one embodiment.

FIG. 3 illustrates how conditioning stimuli may be selected for trainingwhere the user can directly communicate to the device/system, or via anetworked computing device, or may interact with a third party device tochoose training content.

FIG. 4 shows how individualized stimuli may be delivered to the userthrough modular devices in accordance with one variation.

FIG. 5 shows device components for monitoring physiological features ofsleep staging with an electroencephalogram detector (EEG) in accordancewith one variation.

FIG. 6 shows techniques for sleep phase detection and storage of sleepstaging data in accordance with one variation.

FIG. 7 shows how real-time monitoring of biosignals to assess arousalstate and sleep state can be used to choose or modulate the sensorystimuli delivered during SWS in accordance with one variation.

FIG. 8 shows how sensory or electrical stimulation at device-definedfrequencies may be used to enhance or disrupt slow-wave ‘delta’oscillations during slow-wave sleep in accordance with one variation.

FIG. 9 shows methods for determining user-specific training content inaccordance with one variation.

FIG. 10 shows how the invention determines individualized trainingparameters to improve the efficiency of memory training and memoryconsolidation during sleep in accordance with one variation.

FIG. 11 is a schematic illustration of one variation of a memoryenhancing device or system.

FIG. 12 shows an electronic circuit that may be used to drive a piezoatomizer for the variation shown in FIG. 11.

FIG. 13 illustrates one variation of a schematic for a mounting board ofone variation of a device such as that shown in FIG. 11.

FIG. 14 shows an internal view of one variation of a device forimproving memory during sleep as described herein.

FIG. 15 shows a top view of the device of FIG. 14.

FIG. 16 shows a rear view of the device of FIG. 14.

FIG. 17 shows a side perspective view of the portable device of FIG. 14.

FIG. 18 illustrates one variation of a microcontroller for a device suchas the one shown in FIG. 14.

FIG. 19 illustrates one variation of a microSD card that may be used aspart of the device as described herein.

FIG. 20 shows an exemplary computer screen capture showing avisuospatial memory game used in a prototype system, in training modewhen the user is taught the location of each object.

FIG. 21 shows another exemplary computer screen capture showing avisuospatial memory game after the user clicks the location at whichthey remember the target object to have been previously presented.

FIG. 22 is a chart showing exemplary results from the test task using aprototype as shown in FIGS. 14-17 when subjects are trained using amemory game as illustrated in FIGS. 20 and 21.

DETAILED DESCRIPTION

Reference throughout this specification to “one embodiment,” “onevariation,” “an embodiment,” “a variation” or similar language meansthat a particular feature, structure, or characteristic described inconnection with the embodiment or variation is included in at least oneembodiment or variation of the present invention. Thus, appearances ofthe phrases “in one embodiment,” “in on variation,” “in an embodiment,”and similar language throughout this specification may, but do notnecessarily, all refer to the same embodiment or variation.

The devices, systems and methods described herein may be embodied inother specific forms without departing from their spirit or essentialcharacteristics. The described embodiments and variations are to beconsidered in all respects only as illustrative and not restrictive.

Memories formed during wakeful activity are consolidated into long-termmemories during sleep. Certain phases of sleep are especially importantfor memory consolidation, and ambient stimuli that occur during studyingcan facilitate consolidation if replayed during those important sleepphases. We have built a system, described herein, that enhances memoryformation. The system associates the various pieces of information theuser is attempting to learn with the sensory environment experienced bythe user while studying. The system can either record natural stimuli inthe study environment, such as the music playing or other ambientnoises, or it can deliver stimuli, such as pleasant and unique odors orsounds. Later, when the user is sleeping, the system monitors sleepphase by analyzing various sensors such as those for monitoring brainwaves, muscle activity, movement, heart rate, and breathing. In oneembodiment, when the system determines that the user is in a phase ofsleep important for memory consolidation, the system replays the stimuliassociated with the information currently being learned. Reactivatingthe sensory stimuli experienced during an awake study may aidconsolidation of the target information during sleep. In the morning,the system may track learning progress by testing the user's ability toremember the information studied during the previous day andconsolidated during sleep.

The following terms may be used herein. As used herein, the phrase“Training Mode” optionally may be referred to as a Study Mode or theTraining Phase, and may refer to a state of the device or system duringwhich the user utilizes the memory enhancing device during a session tolearn training content. The device activates its stimulus generators atopportune times as the user experiences elements of learning content.Training Mode or Study Mode may occur during a Training Period, StudyPeriod, Training Session, or Study Session.

As used herein the term “Sleep Mode,” optionally referred to SleepConsolidation Mode, refers to a state of the invention, during which theuser utilizes the memory enhancing device during sleep. The device mayassess the user's sleep state. In some variations, during slow wavesleep, the device activates its stimulus generators to enhance memoryconsolidation.

For the purposes of this description, physiology may refer to anybiological process in a human.

As used herein, “memory” or organic memory (as opposed to computer ordevice memory) typically refers to a mental process by which informationis encoded, stored, and retrieved by the brain. There are differenttypes of memory, including motor memory, episodic memory, and emotionalmemory.

As used herein, a mobile device may include a hand-held portablecomputing device (e.g. cellular phone, portable music player, orpersonal digital assistant).

As used herein, “Contextual Sensory Stimulus” may refer to a sensorystimulus that is co-present with elements of training content. Forexample, a green light presented while the lyrics to a song are beinglearned, or the aroma in a room while vocabulary is studied.

As used herein, “Ambient Stimuli” may refer to sensory impressionsexperienced by the user while studying in a natural environment, beyondwhat the system itself produces, such as music, coffee aroma, or busvibrations. “Non-Distracting Sensory Stimuli” may refer to a sensorystimulus which does not disrupt the process of studying, or the sleep ofthe user.

As used herein “Learning Database” may refer to a relational set linkingeach element of information from training content to the appropriatecontextual sensory stimulus.

“Training Content” may refer to a set of facts or information to belearned.

As used herein, “Trigger State” may refer to a stage of sleep or aspectof sleep (e.g. slow wave sleep) that cues the device or systems formodifying memory described herein to begin delivering sensory stimuli tothe user.

As used herein, “Modulating memory consolidation” may refer to enhancingor disrupting memory consolidation, or enhancing and disrupting memoryconsolidation (e.g., enhancing some, disrupting others).

In general, the systems described herein may include one or more of: auser interface that is configured to allow a user (without requiring athird party) to control and operate the system; a sensory stimulatorconfigured to provide a plurality of distinct sensory stimuli; a sleepmonitor to monitor the user's sleep state; and a controller tocoordinate the operation of the system in providing the appropriatesensory stimulus during a training period and again later during one ormore sleep consolidation periods. The system may be specificallyconfigured to be operated by the user over a plurality of differenttraining sessions, providing different (i.e., unique) sensory stimuli tocoordinate with the different training sessions and/or differentmaterials to be learned.

The systems and devices described herein may combine multiple modulesinto a cohesive system. These modules or components may include controllogic (e.g., software, firmware, hardware, or the like) which matcheslearning content with associated sensory stimuli delivered, and hardware(e.g., sensory stimulators) such as scent dispensers or audio speakersand/or recorders. During sleep, monitoring hardware such aselectroencephalography (EEG) for recording brain rhythms may be analyzedby logic (e.g., algorithms or logic for detecting deep sleep, such asslow-wave, or delta, 0.5-4 Hz, rhythms in the EEG signal) or other sleepstages and then triggering presentation of sensory stimuli (e.g., soundsor smells). In some variations, performance may be monitored by thesystem to test memory performance the following day. The platform mayuse wireless and internet-based networking technologies to access adatabase that may include learning content, sensory stimuli, and/orsleep monitoring parameters in order to optimize system performance foran individual user or patient population.

For example, FIG. 1A shows one variation of a schematic illustratingsome of the functional components of a system or device. In thisexample, the system includes a user interface, which has a control toswitch the device to a training mode. The system or device may includevarious modes of operation, including a training mode, during which asensory stimulus may be applied, and a sleep consolidation mode, duringwhich the system monitors the user/patient to determine if the user isin a desired sleep state (e.g., a sleep state conducive to memoryconsolidation). Thus, the system also includes a sleep monitor, examplesof which are provided below, to determine or estimate the patient'ssleep state. The system also typically includes a stimulus generator togenerate a plurality of sensory stimuli, and a controller to coordinatethe activity of the system, as discussed herein. In some variationsthese elements are separate modules or components, which may befunctionally connected (via the controller, for example). In somevariations these elements are completely or partially housed within ahousing, and may form a unitary device.

FIG. 1B shows another variation of a system/device for modulating memoryin a sleeping user. In this embodiment, several functional regions/stepsare shown. The first step is to provide a piece of control logic (e.g.,software) that functions as a desktop, web, or handheld deviceapplication (FIG. 1B). The control logic may permit a user to search adatabase of existing learning content supplied by a third party or theuser may upload or otherwise enter their own information to be learnedinto the database. In another embodiment, the software may selecttraining content for the user based on the user's interests, friends,locations where they have been, membership in a group (such as being astudent in a particular class), or other sources (see FIG. 9, below).

In FIG. 1B, once the user and/or software has selected training contentfor the user to learn, the system may present the user with the trainingmaterial in a manner that can be conducive to learning such as aflashcard-like framework in which one item of a training pair (e.g. atechnical term) may be shown to the user, followed by the other item ofthe training pair (e.g. the definition of the technical term).

At the same time or near in time to the user's learning session, asensory cue can be presented, for instance a sound delivered by aspeaker (one variation of a sensory stimulator), a scent delivered by anodorizer (another variation of a sensory stimulator), a haptic stimulusdelivered to the user's body, or other sensory stimulus (FIGS. 3 and 4).

In some variations, the device may record the training content learnedand sensory stimuli delivered to a database that can be stored on thedevice used for training or remotely via the Internet or an intermediatedevice.

When the user goes to sleep on the subsequent night or a later night orduring a nap, the user's sleep state can be monitored by one or moretechniques appreciated by one skilled in the art (FIGS. 5 and 6).

The system may include sleep monitoring that can identify particularstages of sleep and communicate this information to a device thatdetermines whether to deliver sensory stimuli and, if so, which sensorystimuli to deliver at which sleep stage to a particular user. In oneembodiment, sleep monitoring may be accomplished byelectroencephalography (EEG) and the sleep state during which sensorystimuli are delivered may be deep sleep identified on the basis of oneor more brain rhythms such as delta rhythms (generally about 0.5-4 Hz)and/or the absence of muscle activity related to eye or other movementsgenerally indicative of other stages of sleep. In one embodiment, thequantity or quality (e.g. intensity) of a particular stage of sleep thatmay be deep sleep can be increased or decreased by delivering sensorystimuli or electrical stimulation (FIG. 8). In one embodiment, thechoice of sensory stimulus and/or its intensity and/or its rate ofrepetition can be adjusted based on the level of arousal of the userduring sleep (FIG. 7).

In some variations, efficacy of the system may be determined byre-testing the user the following morning or at a later time todetermine whether training material was learned.

In some variations the system may save information to a memory,including a local memory or database on a base device or remotely viathe Internet or an intermediate device. Stored information may berelated to a) training content; b) the success or failure to remember aparticular item of information; c) sleep monitoring data related tophysiology and/or sleep phase; d) the sensory stimuli delivered and theintensity of the sensory cues (e.g. how loud was a sound); and/or e)other information.

In some variations the system or device uses data mining and/or otherstatistical techniques to analyze stored data in order to improve orotherwise modify the quality of memory enhancement and/or sleepdetection for the user or for other users (FIG. 10). For instance, inone embodiment, the rate of repetition of particular training contentmay be optimized.

In one embodiment, the system or device may include a module (e.g.,stimulus generator) to deliver stimuli during the presentation oftraining content, an element to deduce the user's phase of sleep (e.g.,sleep monitor), a processor (e.g., controller and/or control logic) tocoordinate what stimuli the user should be exposed to, and to cause thestimulus-generating module to create the determined stimuli. The systemor device may be controlled by the user, and may allow evaluation,optimal training material, and storage and creation of meta data relatedto training, stimuli used, sleep, and memory performance in the contextof the training system, and methods to keep an accounting of whichstimuli are associated with what data. The system or device may also usespecific stimuli to modulate the user's phase of sleep.

In some variations, the system or device interacts with the user undertwo primary conditions: an initial learning event whereby an awake userattempts to learn a fact or concept, (training phase), and a latermemory consolidation phase, during which the invention monitors theuser's sleep state, and presents stimuli at specific epochs for thepurpose of improving the user's memory of the initial learning event. Inthe awake phase, a user is presented with learning material. During thisinitial learning/training phase, the device may present a conditioningstimulus, or may record an ambient stimulus for future playback. Later,in the consolidation phase, as the user enters sleep, the device maymonitor physiological parameters to identify sleep state. At varioustimes during sleep, the user may enter a slow wave sleep (SWS) epoch,which may be identified by the device and prompts the device to presentthe conditioning stimulus. In other embodiments, sensory stimuli may bedelivered during another phase of sleep such as light sleep, REM sleep,or specifically during phases S1, S2, S3, or S4 of non-REM sleep.

The sleep monitor may be a subsystem or module that determines theuser's sleep state. For example, in some variations, electroderecordings can be used to monitor signals generated by eye movements(electrooculography; EOG), muscle contractions (electromyography; EMG),or neural activity in the brain recorded from the scalp(electroencephalography; EEG). In other embodiments, sleep phase may bedetermined by one or more measurement devices (sleep monitors) that mayinclude: accelerometers or other devices for detecting body movementwhether directly by a device mounted on the user (e.g. a wristband,headband, stick-on device, etc.) or one placed nearby to detectmovements for instance through a bed, pillow, or cover; devices tomeasure heart rate, heart beat entropy, or other features of anelectrocardiogram (ECG); devices to measure respiratory rate, entropy,or tidal volume of breathing; devices to measure blood oxygen saturation(SpO₂), for instance by a pulse oximeter placed on a portion of thebody; video monitoring of body movements, eye movements, heart beats,breathing, blood oxygenation, or other features detectable by a videocamera; devices to measure body temperature such as a thermometer or aninfrared sensor directed at the user; devices to measure arousalthreshold for instance by presenting a sensory cue (e.g. auditory,haptic, or visual) of known intensity and monitoring variousphysiological parameters that indicate lighter sleep or wakefulness orby delivering a small electrical stimulus targeted to a muscle orperipheral nerve and using a recording electrode to determine thestrength and extent of a reflex response; or any other suitablemeasurement devices currently known or hereafter developed. In yet otherembodiments, multiple techniques for detecting biosignals are used atthe same time and executable logic (e.g., an algorithm) for signalprocessing and statistical interpretation of the signals is used todetermine sleep stage. In another embodiment, standard polysomnography(PSG) techniques are used to determine sleep staging.

The systems and devices described herein may include an element or setof elements to generate a stimulus. Such stimuli are typically capableof activating one or more of a user's auditory, somatosensory,olfactory, gustatory, visual, vestibular, or other sensory systems.Examples of stimulus generating elements include a screen or light,speaker, tactile electrical or mechanical stimulator, or scent-releasingdevice.

A learning event may be defined as the presentation of a fact-stimuluspair controlled by a computer training system, computer or Internetsite, or the invention; a third party controlled moment or exposure suchas a real life learning or experiential training moment, an interaction,or most generally, any event during which an aspect of the event is tobe recalled or utilized at a later time. For example, when a person isstudying the Fourth Amendment of the U.S. Constitution, they hear thesound of a violin or a song. In some embodiments, the fact and sound arenow linked as a pair. Information regarding learning events are storedby the invention for subsequent analysis and future presentations of thefact-stimulus pair. Such presentations constitute the training phase.

The systems and devices described herein enable learning material to becompiled for a user from several sources that may include: 1) materialentered into a computing device by the user or a third party that may bestored locally on the device or stored on a remote server orintermediate device; 2) material that may be in a variety of electronicformats that can be uploaded by the user or third party to a computingdevice for storage through the Internet on a remote server or on anintermediate device; 3) material chosen by the user from apre-determined set of training material that may be provided by a thirdparty or the device; 4) material that may be generated based on theuser's location at a particular time or ‘check-in’ by a service havingfunctionality similar to that of Gowalla or Foursquare; 5) materialgenerated based on the user's interests as indicated by the user, by athird party, or by data mining; 6) material that may be supplied by athird party such as a teacher, work colleague, friend, advertiser, orother individual or entity; or 7) material supplied by other suitablemeans appreciated by one skilled in the art, that may be currently knownor hereafter developed.

As mentioned above, in some variations, the learning stimulus can be anauditory, haptic, visual, gustatory, olfactory, or electrical stimulus.The invention may host a selection of stimuli that can be used by theinvention or by third party devices during the learning phase. Forexample, a professor's presentation could use embedded audio content asa learning stimulus while lecturing to the entire classroom. Such afeature would permit all persons present in the classroom to experiencefact- or content-associated stimuli for later use during sleep phasesassociated with memory consolidation.

In some variations, ambient, rather than provided stimuli may be used.An ambient or observed stimulus may be recorded during thelearning/training event. In situations where the environment is rich instimuli suitable for re-presentation during the sleep consolidationphase, or situations where the user is unable to receive createdstimuli, the system may record one or more of the environmental stimuli,for re-presentation. This information can be transmitted to the sleepstimuli generator, another device, or to a server (e.g., stored in localor remote memory for later presentation). Any one of these devices mayparse the stimuli or refine it according to signal processing orstatistical interpretation techniques to make it suitable forconsolidation during the sleep phase. For example, in the case where aperson is studying in a café the device, which could be a program on amobile device, will record audio through a microphone while the user islearning. Later, the audio can be filtered to find epochs of therecording where the audio has more variance than other portions of thelearning period or times of the day when the user is not using thedevice for learning.

In some variations the system or device may synchronize with and/orharvest from third party devices or services. In one embodiment,services that provide streaming audio content can be linked to the userby their geographical location or by using or being logged in on such aservice like an Internet radio service. A listing of stimuli the userwas exposed to (e.g., a playlist) can be parsed to correlate stimuliwith learning events. The device or system may then replay these stimulifor training during the sleep consolidation phase.

In a classroom setting, in which multiple students may attend a classwhere material is being taught and stimuli are presented, the system ordevice may apply an ambient stimulus to generate stimulus-content pairsto be recorded to a device/system locally or remotely (e.g., userdatabase accessed through the Internet on a remote server), or throughan intermediate device. The system or device may support ways that theuser can implicitly connect these learning events to their accounts. Forexample, a user can register attendance at such an event, or, forinstance, museum exhibit, by using a QR code, hashtag, or hyperlink,registering with an RFID or near-field communication device, or bychecking in with a third party or socially-derived network with thefunctionality of a service such as Foursquare or Gowalla. The elementmay include a unique event identification (ID) or a combination ofgeographic and temporal information to ensure the appropriate event isregistered.

In some embodiments, the information to be learned is directlyvisualized by the use of a computing device such as a desktop computer,tablet device, mobile device or the like. The system or device may beaccessed as an Internet browser application, or as a standaloneapplication. In both cases, the invention may be responsible forpresenting training stimuli, such as visual cues on the computingscreen, or auditory cues presented on speakers.

In some embodiments, a secondary user may access information regardingthe primary user's use through the Internet. In other embodiments, thesecondary user may remotely initiate the learning phase. For example, ateacher may remotely access a student's use for the purpose of trackingprogress made on a lesson rubric.

In some embodiments, the invention is coupled with the use of electronicbook readers. This embodiment may focus on learning a new language, oracquiring new vocabulary. The system or method may interface viasoftware to the electronic book reader, or via an external device thatcommunicates with the electronic book reader.

In some variations, the system/device is applied to study for K-12government-mandated tests, other standardized tests related toapplication to academic institutions test preparation for standardizedtests such as those required for admission to primary or secondaryschools, universities, post-graduate programs, or other academicpursuits; tests required for admittance to a specific professional groupincluding but not limited to the state bar exam for lawyers, theCertified Public Accountant (CPA) exam for accountants, the medicalboard exams for doctors, the Series 7 exams for brokers, or otherprofessional tests or exams that would be known to one skilled in theart of test preparation.

The components of the systems and devices described herein may beembedded in mobile devices, either in the form of hardware and/orsoftware running on general purpose devices such as cellular phones,portable music players, tablet computers, or other suitable portabledevices currently known or hereafter developed. A mobile device mayprovide data connectivity to other networked parts of the memorylearning system. For example, the user interface for controllingpriority of learning material can be controlled via software running ona mobile device. Additionally, because mobile devices are alreadycarried by many people, they can be used to ‘check in’ to locationswhere learning content has been, is being, or will be presented to theuser. Also, ambient sounds or other sensory stimuli recorded by themobile device could be used to generate stimuli for re-presentationduring the memory consolidation phase. These recorded stimuli could bepresented as recorded or processed on the mobile device, a remoteserver, or an intermediate device to extract features that are salientor otherwise beneficial to training. Additionally, mobile devices canprovide data display of learning content, training programs, stimulipatterns previously presented or to be presented during sleep, or othercontent. The mobile device can be used to generate sleep quality queriesor other subjective ‘rate yourself’ style polling; to present remindersto complete training; to generate audio or haptic stimuli; to controlother devices that deliver training content or sensory stimuli. In somevariations a mobile device may be configured to monitor sleep staging;to receive and/or process data about sleep staging from another device;or to function as a bedside unit to record EEG or other physiologicalsignals sent wirelessly from a device worn by the user or placed nearthe user.

Thus, in one variation, the system is configured to be executed as anapplication on a handheld device such as a mobile phone (e.g., iPhone,Android, iPad, etc.). The application may include control logicconfigured to run on the mobile device and record and/or apply a sensorystimulus during the learning/training phase. The control logic maypresent a user interface (e.g., a control operated through the mobilephone's touchscreen or buttons) to indicate when the training mode is tostart and presentation and/or recording of the sensory stimulus (e.g., asensory stimulus generated by the mobile device or a peripheralcomponent controlled by a mobile device or ambient sensory stimulus) isto start. Thereafter, the control logic may control monitoring of asleep monitor to determine when the user has entered into a specifiedsleep stage. The sleep monitor function may also be performed by themobile device. As mentioned, the mobile device may communicate with aseparate or integrated sleep monitor (electrode, accelerometer, etc.).In some variations the controller/control logic communicates with asleep monitor module including sleep monitoring logic configured tomonitor the users sleep state. The control logic may thereafter triggerthe repeat of the paired sensory stimulus.

In one embodiment, the system/device is designed to autonomously collectinformation without requiring explicit behaviors or actions from theuser and furthermore to populate activities of the user with contentrelevant to the interests of the user and the capabilities of the sleepconsolidation module. The system/device may use or include easilyembeddable web and local or desktop widgets that can be added towebpages with methods appreciated by one skilled in the art. The devicesmay use implicit behavior such as cursor position, eye gaze, time onwebsite, links clicked, and other web-user interactions appreciated byone skilled in the art to identify meaningful content. Furthermore, bycoupling with third party services that represent information aboutsocial contacts of the users, or other unknown users that share similarindicators of educational interests, the device can make meaningfulsuggestions of content that is related or more easily trainable. Thesecontent exposures can be recorded by the widget into the sleep trainingdevice database to improve the rate of memorization. The widget is oneexample of a method or web element that can be easily included on anyblog, news service, learning service, or information provider. Includingother embeddable web technologies would function in an identical manner.

In one embodiment, the device or system may be used to alter sleepstate. Sleep stage can be altered in several ways including: increasingor decreasing the frequency of a particular sleep state, increasing ordecreasing the duration of a particular sleep state, or altering theintensity of brain rhythms associated with a particular phase of sleep.In the latter embodiment, methods for inducing changes in the brainrhythms associated with a particular phase of sleep could includeelectrical stimulation of the scalp, for instance at delta brain rhythmfrequencies (generally between 0.5 and 4 Hz) with a pair of electrodesplaced on the scalp.

As mentioned above, human sleep is generally described according to acycle in which rapid eye movement (REM) sleep is followed by non-REMsleep that generally proceeds sequentially through phases S1, S2, S3,and S4. Phases S1 and S2 are generally referred to as light sleep, andphases S3 and S4 are generally referred to as deep or ‘slow-wave’ sleep.Light sleep can be identified by examination of body movements, such ashypnic jerks, lack of eye movements, and sleep spindles recorded by EEG.Deep sleep demonstrates enhanced power in the delta spectrum (generallyabout 0.5 to 4 Hz) combined with lack of motor activity. During REMsleep, apart from rapid eye movements, electroencephalogram (EEG)recordings show enhanced power in the theta bands (generally about 4 to8 Hz).

Detecting optimum cognitive states for stimulus driven memoryconsolidation may be done many different ways. Each method is assessedfor the accuracy of sleep phase determination, cost and computationalrequirements, and the burden of compliance placed on the user.Therefore, the system may be adapted to include one or more of thefollowing techniques to determine sleep state: EEG; electromyography(EMG); electrooculography (EOG); motion during sleep (called actigraphymeasured by image capture, accelerometer, microphone, or othertechniques); heart rate, via accelerometer, pulse oximetry, ECG;respiratory rate, via accelerometer, microphone; and body temperature,via temperature probe or distant infrared (IR) sensor.

By applying machine learning and clustering techniques to look atprevious training data for a particular user, or population of users,the rate of memory consolidation and the resolution of sleep phasedetection can be optimized for a user or users. To classify such signalsany of the following methods may be used, including, but not limited to:Independent Component Analysis; Principal Component Analysis; LatentDirichlet Allocation; Naive Bayesian Classifier; K-means clustering; andNeural networks.

A device user interface may allow direct user control over stimuliexposures during the sleep phase. For example, this interface may be onthe device or remote, such as on a company server, mobile device, nearbycomputer, or through a third party content provider, such as a collegeor test preparation service. The user interface may therefore allow theuser to select groups of learning events that they want to be retrainedon. The user or third party can alter meta data and record theimportance of this information for them. The user may have opportunitiesto interact with other users to gain information or context into whatshould be learned. For example, if a user is studying for an upcomingtest they can log into a website and search and select for relevantcontent. That information may be transmitted to a training programcontroller.

The system may handle the accrual of many learning events during use ofthe system, and may therefore use multiple specific and/or unique (to aparticular learned subject and/or training session). For learnedmaterial, or material created solely for evaluation, the user may betested on their retention of past learning events in order to establishthe performance of the platform, algorithms, and stimulation parametersincluding stimulus strength, phase of sleep, maximum number of stimuliduring sleep, optimal times of day for the learning parameters, theneurophysiologic classification of memory type, or any meta information.The results of these tests can be analyzed through clustering or datamining methods to understand how the platform should function and theuser should be trained in the future. Trained sensory stimuli may berepeated immediately after the training/learning phase (e.g., during thenext detected specified sleep stage) or repeated over multiple sleepstages in a single night or over multiple nights. The replay of a pairedsensory stimulus may be delayed for a day or more (e.g., three days, aweek, ten days), or repeated on this schedule. Multiple sensor stimulimay be repeated in a single specified sleep stage, or a single sensorystimuli corresponding to a training phase may be repeated during asingle specified sleep stage.

In general, the systems or devices described herein may also includelogic for efficient memory training and memory consolidation duringsleep. In some embodiments, an additional aspect of feedback is used tooptimize (or improve) the frequency of repetition by recording on thedevice itself or through the Internet on a remote server, or through anintermediate device. Established techniques for optimizing therepetition time in a spaced repetition implementation may be based ondetailed mathematical models of learning and may take into account theopportunity cost of forgetting and content repetition. These algorithmscan be used to improve the performance of the present invention for aparticular user, subset of users, or class of users defined by age,gender, cognitive ability, interests, or any clinically relevant causeof cognitive dysfunction. In various embodiments, optimization can beapplied to various components of the present invention, including therate of repetition of training stimuli, the amount of learned materialor specific set of content associated with a particular stimulus orclass of stimuli, the salience of stimuli associated with trainingstimuli, or other aspects.

Optimum learning strategies based on the above sleep-based memoryconsolidation may be individualized. Parameters of the sensory stimuliassociated with learned content and sleep may affect the quality ofmemory consolidation. These parameters may include the choice ofstimulus cue, timing of stimulus delivery, quantity of learned materialassociated with a particular cue, and the modality, intensity, oremotional valence of the sensory input. One method for identifyingoptimum parameters would be to use a collaborative application thatenables interaction among students, teachers, and content-providersthrough the Internet on a remote server, or through an intermediatedevice. Groups of learners may be able to share their experiences andbest-parameters. Additionally, learning experts or third-party servicescould interface with these groups, providing advice, instructions,and/or content.

EXAMPLES

FIGS. 11 to 19 illustrate one exemplary variation of a device forenhancing memory during sleep. In this variation, the device includesseveral components. The device may include one or a plurality of userinterface components that allow a user to control whether the device isin training mode. The user interface is generally configured so that thedevice may be controlled by the user without requiring additionalassistance. The user interface may also allow the user to select otherparameters of device function. The device may also include one or aplurality of stimulus generators (optionally referred to as stimulusactuators) capable of activating at least one sensory transductionpathway. The device may also include one or a plurality of sensors thatmeasure user physiology to determine the sleep state of the user. Thedevice may include logic that estimates the sleep state of a user fromrecorded physiological and other data. The device may also include acontroller comprising control logic that determines the device functionbased on user inputs, the user's sleep state or wakefulness, andprevious device use cases by the user.

In some embodiments, the device also includes one more outputs (useroutputs) such as screens, light emitting diodes (LEDs), or othercomponents to indicate device function. The device may also include oneor more switches or other control elements for the user to controldevice function. In some variations the device includes acomputer-readable/writeable memory component (local or remote). In somevariations, the device includes a controller and control logic; thedevice may also include send/receive sub-systems for transmission ofdata between the device and an off-site computer.

Example 1 A Portable Memory Enhancing Device

In one embodiment, such as that shown in FIGS. 11 to 19, the device isconfigured as a lightweight, portable device that includes a plasticenclosure or housing. In the example shown, the housing is a blackplastic case made by B&W International (e.g. Outdoor Case Type 05). Inthis embodiment, the device includes a single user interface component,including a two-position, three-pole switch mounted on the top of theenclosure. The default switch position is ‘Sleep Mode’. To enterTraining Mode, the user toggles the switch. The switch reads either 5volts or ground and is connected to a digital input pin of amicrocontroller contained in the device. The software installed on themicrocontroller reads the state of the switch and sends appropriatecontrol signals for either Sleep Mode or Training Mode. See below forfurther details about the microcontroller and the control logic achievedby software installed on the microcontroller.

In this exemplary embodiment, the device contains two stimulus actuatorsfor generating scents that activate olfactory transduction pathways inthe user. The scent-releasing mechanism employs a piezo-actuatorcontrolled by an appropriate electrical circuit to atomize a volatileliquid held in a reservoir below the atomizer and coupled to it by awick. The electronic circuit used to appropriately activate thepiezo-actuator is shown in FIG. 12. Digital control pulses oscillatingat or near the resonant frequency of the piezo-actuator (1202) are sentto the electronic circuit. High pulse values to transistor (1204) allowcurrent supplied by DC power (1201) to flow through one coil of thetransformer (1203), in turn inducing currents to be sent to the piezo(1206). An inductor (1207) smooths the induced current traces withoutaltering the oscillating frequency to generate an effective waveform fordriving piezoelectric oscillation and thus releases a scent. Ground isshown at 1205.

In one variation, the circuitry, piezo-atomizer, and odorant may besimilar to those described in U.S. Pat. No. 6,439,474 B2 titled “Controlsystem for atomizing liquids with a piezoelectric vibrator” by inventorDennis J. Denton, herein incorporated by reference in its entirety.

In this embodiment, the sleep monitor used to determine the user's sleepstage derives sleep stage from brain rhythms recorded byelectroencephalography (EEG). In this example, a pair of conductivefabric recording electrodes is mounted on a headband worn by the userduring sleep. The recording electrodes are oriented on the user'sforehead and register both brain rhythms and electrical signalsgenerated by contraction of facial musculature, such as eye movementsknown to occur during rapid eye movement (REM) sleep. The wearable EEGsystem includes a battery and electronic circuits to amplify, filter,and process the recorded signals. A wireless communication protocol(e.g. Bluetooth, ANT, or another wireless communication protocol) may beused to transmit the raw and/or processed signals to an EEG receiverboard component of the device.

Signal processing of EEG signals may use Fourier decomposition (alsoreferred to as a Fourier transformation) or another effective algorithmto quantify the relative and absolute proportion of signal powercontained in various frequency bands. Other algorithms can be employedto reduce noise, reduce or eliminate movement artifacts, and modifyother features of the recorded signal that do not relate to thebiological signals of interest generated by the brain and otherexcitable tissue such as facial or ocular muscles. Various mathematicalprogramming libraries facilitate simple testing and application ofsignal processing algorithms. For instance, Python, Matlab, LABView, orother software packages.

Sleep state logic may be executed on a processor (e.g., microprocessor)to determine from the recorded EEG data the sleep stage of the user. Thelogic may analyze data that extends over at least about 100milliseconds, at least about 1 second, at least about 10 seconds, atleast about 50 seconds, at least about 100 seconds, at least about 5minutes, at least about 10 minutes, at least about 30 minutes, orlonger. By analyzing data that extends over time, signals can beaveraged to increase the signal-to-noise ratio. Various well-knownstatistical and signal processing techniques can be used for averaging,weighting signals of interest, enforcing thresholds, and applyingvarious heuristics to distinguish specific stages of sleep such as lightsleep, deep (slow-wave) sleep, REM sleep, and wakefulness. For instance,delta rhythms occur during slow-wave sleep. High amplitude, highfrequency signals generated from facial muscles and eye movements occurduring REM sleep. Heuristics can also be incorporated into the sleepstage processing algorithms that take into account the known order ofsleep stages during the sleep cycle. The user's sleep state may bedetermined and used by the control logic system (see below).

A headband and signal processing unit may be charged by placing the uniton a docking station mounted on the top of the enclosure. The dockingstation may be connected to an EEG PCB board that includes varioussignal processing and wireless communication functionalities. Placingthe headband unit in the docking station matches the headband unit tothe EEG PCB board for subsequent wireless communication. The EEG PCBboard in this example also includes a serial communication port thattransmits a serial stream of data corresponding to the user's sleepstate, time stamps, and other information. The serial stream is receivedand parsed by the programmable microcontroller.

In this embodiment, a programmable microcontroller board applies controllogic that determines device function based on user inputs, the user'ssleep state or state of wakefulness, and previous device use by theuser. For example, an Arduino open source microcontroller framework isan effective programmable microcontroller board used in this embodiment.The Arduino system includes digital inputs and outputs, analog inputsand outputs, serial receiver, serial transmission, and power (both 5Vand 3.3V). The microcontroller system in this example is programmed withcustom software for controlling the various elements of the system formemory enhancement.

In one specific embodiment, the microcontroller begins by first checkingwhether the device is in Training Mode or Sleep Mode. If the device isin Sleep Mode, the microcontroller begins reading information via aserial communications receiver in real-time from sleep-phase detectioncircuitry. When the user puts on the EEG headband unit, an LED indicator(the headband LED indicator, LED1) is turned on by changing theappropriate digital output of the microcontroller to a high (5V) state.The microcontroller parses sleep stage information for the user andstores these data with timestamps in a memory component of the device.If the incoming sleep data indicates the user is in a slow-wave sleepepoch, scent delivery logic is activated by changing the appropriatedigital output to a high (5V) state. The activation of the scentdelivery logic is also registered on the memory component of the devicewith a timestamp. While slow-wave sleep is occurring, the time ofoperation is identified and compared to a training schedule, such thaton appropriate days, scents are delivered. The device loads, reads, andparses a user configuration file stored on the device memory todetermine the appropriate device function based on the training schedulefor the user. The selection of which scent to deliver and the quantityof scent delivered is also determined by the training schedule. An LEDindicator, stimulus indicator (the scent LED indicator, LED2), is turnedon when the scent is delivered during sleep by changing the appropriatedigital output of the microcontroller to a high (5V) state. The LEDindicator can be left on for the remaining portion of the night so thatthe user can observe it upon wakening. In such embodiments, the controllogic turns off LED2 at a fixed time (e.g. 2 hours) after the userwakes. When the slow-wave sleep epoch ends as determined by theregistration of a different sleep state by the sleep state monitoringcomponents of the system, the scent delivery logic is changed toinactive and scent release ceases. While the subject is in non-slow wavesleep states, scent delivery logic is not active. If the device is notbeing used during sleep, the microcontroller does not acquire sleepinformation from the associated EEG hardware and no scents are released.

If the device is in Training Mode, the microcontroller digital outputsare changed to high (5V) for LED2 and the appropriate scent deliveryunit as determined by the training schedule for the user. At the end ofa Training Mode session, the user toggles the position of the userinterface switch and the microcontroller responds by changing theappropriate digital outputs to turn off LED2 and cease scent delivery.

A schematic of this embodiment is shown in FIG. 11. In FIG. 11, DC power(5V, 1102) and ground (1101) are used to power the electronic componentsof the device. The programmable microcontroller (1110) includesconnections for a plurality of digital outputs (1108), one digital input(1112), a serial transmitter (1111), and a serial receiver (1113). Thedigital outputs are used to control device components: LED1 (headbandLED, 1105); LED2 (scent delivery LED, 1104); a memory storage component(1103); and two piezo atomizer units for delivering olfactory cues (1106and 1107). An in-line resistor of about 100 ohms (1109) is used for eachof the LEDs. The user (1116) wears a headband that includes EEGelectrodes and an EEG amplification and signal processing unit (1115).EEG and sleep state information are transmitted wirelessly to an EEGreceiver and (further) signal processing unit (1114) that connects tothe serial receiver pin (1113) of the microcontroller. The state of thedevice (Training Mode vs. Sleep Mode) is determined by the position of aswitch (1117) that is connected to a digital input of themicrocontroller (1112) which reads either ground or 5V. In someembodiments, a computer or other serial monitoring terminal (1118) isused to monitor device function via the serial transmitter pin of themicrocontroller (1111) for monitoring, debugging, and/or testingpurposes.

In this embodiment, a custom designed mounting board (FIG. 13, 1301) isused to align and mount the various components. A form is used foralignment to drill holes of various diameters in the enclosure formounting screws and placement of LEDs, scent ports, a power socket, andthe switch. The mounting board includes pre-drilled holes for mountingscrews (or other appropriate hardware) (1302). Next, the microcontroller(1308), two piezo-atomizer circuits (1305, 1306), and the EEG PCB board(1307) are mounted onto the custom mounting board (1301). The two LEDs,switch, power socket, and two piezo-atomizers (1303, 1304) are mountedto the enclosure with appropriate mounting hardware. In FIG. 13, theposition of the two LEDs, power socket, and switch are not shown. Next,the mounting board is installed with appropriate mounting hardware,including spacers. Power plugs are connected to the EEG PCB board andmicrocontroller. Hookup wire or other soldered connectors are used toconnect all components as required.

Example 2 A Memory Enhancing Device with a Computer-Readable MemoryComponent

Some embodiments include one or a plurality of memory storage componentsthat permit data to be read from and written to computer-readablememory. In one embodiment, data is stored on a microSD card coupled tothe programmable microcontroller for offline analysis and confirmationof device function. The stored data can also be used in case of powerinterruption to determine what stage of an experimental protocol a userhad reached.

One advantageous feature of embodiments with data storage components isthe capacity for the system to store data about the user's sleep and thetimes when: a user interacted with a user interface component; the stateof or information presented by an indicator on the device was changed;and/or a sensory stimulus was generated. For instance, the device canwrite to the memory components if a switch position was changed; if anLED was turned on or off; if a stimulus such as a particular scent wasreleased; and when a user's slow-wave sleep epoch started and ended. Inadvantageous embodiments, for each memory storage event, a timestamp mayalso be written to the memory storage. In this embodiment, storage ofdata about user physiology, user interaction with the device, and otherinformation can be used in real-time or for post hoc analysis in orderto improve performance of the device and/or to provide feedback to theuser, for instance about their memory performance over time.

Another advantageous feature of embodiments with data storage componentsis the capacity for the system to read configuration files thatinfluence the control logic output of the device and thus devicefunction. In some embodiments, configuration files are general-purposeand indicate a particular training or studying regimen for a user.

In one specific embodiment, the system writes to a configuration fileeach time the user completes a particular phase of use of the device,including when: (1) a user uses the device in Training Mode for at leasta minimum amount of time. The minimum amount of time may be about 10seconds, about 30 seconds, about 1 minute, about 10 minutes, or longer;(2) an epoch of slow-wave sleep is detected in a user when the device isoperating in Sleep Mode. Each of these events can be registered on eachday the device is used.

In some embodiments, the memory storage components of the device arelocal to the device for direct (wired) communication or wirelesscommunication via a wireless protocol such as Bluetooth or ANT. In otherembodiments, the memory storage components are present on a remoteserver that is accessed via the Internet.

Example 3 Alternative Embodiments of Memory Enhancing Device

In alternative embodiments, the enclosure of a portable memory enhancingdevice is made of plastic, wood, metal, or another suitable material.Other embodiments of a memory enhancing device use one or morealternative or additional user interface components that allow the userto control device function, chosen from the list of: touchscreen,buttons, switches, and other mechanical or electrical input or selectionmechanisms. In some embodiments, the memory enhancing device includesone or a plurality of screens, light emitting diodes (LEDs), or othercomponents to indicate device function by visual, auditory, tactile, orother means. These embodiments are advantageous for indicating to theuser the device mode, history, options, and other information.

In some embodiments, alternative methods for releasing a scent such asan aerosol spray or another technique known to those skilled in the artof generating smells can be used. In further embodiments, a plurality ofscents can be delivered to the subject. In embodiments capable ofdelivering multiple odorants, distinct scents can be generated byreleasing multiple scents concurrently or within a sufficiently shortperiod of time that the volatile compounds are present concurrently inthe user's environment. In such embodiments, the ratio of the two ormore scents released can be varied to generate a larger number ofdistinct olfactory percepts.

In alternative embodiments, the stimulus actuators generate an auditorystimulus. For instance, the auditory stimulus can be generated byspeakers contained in the device. Alternatively, the auditory stimuluscan be generated by headphones connected to the device with wires or viaa wireless protocol such as Bluetooth. In some embodiments, theintensity of the auditory stimulus is adjusted based on the currentsleep state of the user, such that the stimulus does not disrupt theuser's sleep. In some embodiments, physiological measurements of sleepstage and/or sleep depth are used to adjust the intensity of auditorystimuli. A similar strategy of changing stimulus intensity as a functionof sleep state of the user can be used for other modalities of stimulias well.

In alternative embodiments, the stimulus actuators generate a tactilestimulus. A tactile stimulus can be generated by a wearable componentattached to the user such as with a headband, wristband, piece ofclothing, fashion accessory, or other means of coupling physically tothe user. The tactile stimulus can be generated by any means thatactivates somatosensory transduction pathways for instance by activatingperipheral receptors that mediate touch, temperature sensation, or pain.In various embodiments, tactile stimuli can be generated by a piezoactuator, buzzer, heating element, or other mechanism known to oneskilled in the art. Examples of non-contact tactile stimulus actuatorsinclude ultrasound transducers, components that generate a magneticfield, and other non-contact means to activate somatosensorytransduction pathways.

In alternative embodiments, the stimulus actuators generate a visualstimulus. Advantageous visual stimuli are those that are low resolutionand thus could be effectively transmitted both to an awake user inTraining Mode and a sleeping user in Sleep Mode. Advantageous lowresolution visual stimuli contain positional and light level informationbut are not intended to transmit high resolution information. Forinstance, a white or colored light in a particular area of the visualfield such as the left upper visual field. In some embodiments, redlight is used as an advantageous color due to its relatively highertransmission through closed eyes relative to other wavelengths of light.In some embodiments, visual stimuli are time-varying. Temporal patternsmay be regular (e.g. sinusoidally varying light intensity), irregular,random, or pseudo-random. Visual stimuli may be generated by one or aplurality of LEDs, an OLED screen, or other controllablelight-generating source. The visual stimulus actuators may be mounted oneyeglasses, a hat, or other wearable forms of clothing or accessories.

In alternative embodiments, the stimulus actuators generate a gustatorystimulus. In such embodiments, a taste can be delivered via appropriatehardware coupled to the device that delivers tastant molecules to thelips, tongue, palate, and/or mouth. In one such embodiment, anorthodontic retainer securing to the teeth and palate, additionallycomprises microfluidic chambers holding concentrated volumes of tastants(e.g. essential oils, artificial sweeteners). Upon receipt of a wirelesscontrol signal, a chamber releases its contents, which come in contactwith taste buds on the palate and tongue.

In some embodiments, a plurality of stimulus actuators is used todeliver stimuli chosen from two or more sensory modalities. In suchembodiments, a more diverse set of stimuli can be generated, permittinga larger number of unique stimuli to be delivered to a user duringextended, repeated, or ongoing use of the device.

Example 4 Instructions for Use of a Memory Enhancing Device

In this example, the user may be provided with instructions for useincluding several steps that will be described in instructions.Instructions may be written on paper, electronically, or may bepictorial (e.g., ideograms) or the like.

For example, the user may be instructed to place the memory enhancingdevice at a convenient bedside location such as a nightstand and toplugs the power supply into a wall socket. In this case, the user willstudy while sitting in bed near the device. Alternatively, the user mayput the memory enhancing device near them during study at an alternatelocation at home (for instance at a desk or table) or at a cafe,library, or other suitable location for study. In this use case, theuser will need to remember to bring the device to their bedside beforegoing to sleep.

In a second step, the user commences a study session by toggling theswitch to Training Mode. When Training Mode is selected, control signalsare generated by the microcontroller to generate stimulus (e.g., arelease a scent) and turn on a first light emitting diode (LED1) mountedat the top of the enclosure, indicating that the stimulus has beendelivered (e.g., scent releasing mechanism is activated). In one exampleof a study to examine the efficacy of the device (discussed in detailbelow), the user may be asked to study a particular set of content orplay a web-based brain game that tests memory function. The TrainingPhase may last for an appropriate amount of time, for example, fromabout 2 minutes to about 2 hours, from about 2 minutes to about 1 hour,from about 5 minutes to about 45 minutes, from about 5 minutes to about30 minutes, from about 5 minutes to about 15 minutes, etc.

At the end of the training phase, the user moves the switch to the SleepMode position. The scent generation ends and LED1 is turned off. In somevariations the system may automatically move the switch to the sleepmode position or simply change the mode of the system internally. Forexample, the system may be switched to the sleep mode (or sleep phasemonitoring mode/sleep consolidation mode) by a timer or by detectingwhen the subject is sleeping.

The user may be instructed to wear the sleep monitor (e.g., headband)when they go to sleep. The device may indicate when the headband issuccessfully recording and transmitting to the device by turning on asecond LED (LED2) mounted on the enclosure. In one example, the stimulus(e.g., scent) presented during an earlier training session will bepresented again while the subject is in an epoch of deep (slow-wave)sleep. The user will not be consciously aware that the scent ispresented during sleep. They will not be awoken. For the first deep(slow-wave) epoch of the night, LED1 is turned on and remains on untilafter the user wakes up. Thus, in Sleep Mode, LED1 indicates to the userupon awakening whether the scent was released during sleep.

At the end of the night, the user may place the headband back on thedocking station for charging.

Example 5 A Memory Enhancing System Based on an iOS Device (or OtherMobile Device)

In one embodiment, the component functions are achieved by a third partymobile device such as a cellular phone, portable music player, orpersonal digital assistant. In this embodiment, a custom applicationrunning on the mobile device coordinates the components to produce thememory enhancing effect.

In this embodiment, the user interface component is the touchscreeninterface, keyboard, or other user interface of the mobile device. Forinstance, the user selects the desired Mode (Training Mode or SleepMode) using the touchscreen, keyboard, or other interface of the mobiledevice. In some embodiments, the screen of the mobile device can be usedto indicate device function such as whether the device is in TrainingMode or Sleep Mode.

In this embodiment, the mobile device also functions as a stimulusgenerator. Most modern mobile devices contain speakers and/or aheadphone jack, as well as the requisite hardware and software togenerate arbitrary auditory stimuli. In alternative embodiments, themobile device can control the delivery of other modalities of stimulithrough peripheral accessories that can be activated via wiredconnections and wireless protocols such as Bluetooth or ANT. Forinstance, a stimulus generator capable of delivering one or moreolfactory stimuli could connect to the mobile device and be activated atthe appropriate times based on the control logic of the customapplication running on the mobile device.

In this embodiment, the mobile device can also be used as a component tomeasure user physiology that corresponds to sleep state. Many mobiledevices include accelerometer and/or gyroscope components that measuremovements of the mobile device and can be accessed by applicationsrunning on the mobile device. By placing the mobile device on the bed,user movements during sleep can be measured to acquire the necessarydata to estimate sleep states according to the well-establishedtechnique of actigraphy. Similarly, the mobile device includes amicrophone that may be used for monitoring sounds arising from bodymovements, as well as those arising from user breathing patterns. Bothtypes of sounds may be used to assess sleep state. In alternativeembodiments, the mobile device can be used to acquire otherphysiological data related to sleep state either directly or viaperipheral or accessory components.

In this embodiment, the algorithm for determining sleep state based onrecorded physiological data is a component of the software applicationrunning on the mobile device. Most modern mobile devices includehardware components with sufficient memory and processing power to runthe requisite algorithms in near real-time.

In this embodiment, the controller logic that determines device functionbased on user inputs, the user's sleep state or wakefulness, andprevious device use cases by the user is also achieved by the customapplication running on the mobile device. Mobile devices also containbuilt-in memory for storing data about the user, the device function, orother data required for proper memory enhancement using this device.Mobile devices have wireless transmission capabilities for sending thisdata to another computer or remote server via the Internet.

When this embodiment is placed in Training Mode, the device delivers anappropriate contextual sensory stimulus. In some embodiments of themobile device version of this invention, the application also presentsTraining Content on the screen of the mobile device, includingappropriate user interface components and functions. In this embodiment,as the user studies the lesson, the device produces the contextualstimuli.

In Sleep Mode, set by the user prior to sleep onset or activatedautomatically based on the signals derived from accelerometer and/orgyroscope components of the mobile device, the mobile device loads theappropriate contextual stimuli from the learning database and beginscollecting sensor data related to the sleep physiology of the user. Themobile device may collect this data using built-in sensors such as anaccelerometer, gyroscope, microphone, or ambient light level detector,or communicate with accessory sensory devices. The mobile device appliesan appropriate algorithm to the collected data to estimate the currentsleep state of the user. When the sleep state matches the trigger state(e.g. slow wave sleep), the mobile device generates the appropriatecontextual stimulus.

To assess the effectiveness of a particular embodiment of the device, itis advantageous in some cases to use a test or other assessment ofcognitive function, learning, and/or memory. Such tests can take manyforms, including standardized tests, assessments of different cognitivedomains (e.g. language, math, reasoning, foreign language, history,motor performance, and motor learning), tests administered by a trainedprofessional (e.g. a teacher, school counselor, or clinical socialworker), self-administered tests, computer- or web-based brain games,and other formal or informal assessments that can be tracked,quantified, and/or normalized.

One specific example for testing a user's memory employs a visuospatialgame played via the Internet by using a web browser. The visuospatialgame operates in a training phase and a testing phase. In the trainingphase an image is displayed near the center of the screen. After aboutseveral seconds, a target is displayed elsewhere on the screen for aboutanother several seconds. Next, the image and target disappear and thebackground is displayed for about several seconds. The presentation ofimage and target (with a different, randomized or pseudo-randomizedlocation) are repeated for different images. In various embodiments ofthe game, more than about 2 image-target pairs are presented, more thanabout 5 image-target pairs are presented, more than about 10image-target pairs, more than about 20 image-target pairs, more thanabout 50 image-target pairs, more than about 100 image-target pairs, ormore image-target pairs.

In the testing phase of the visuospatial game, the set of imagesdisplayed for a particular user during a training session are presentedagain. The user is instructed to use the mouse or touch screen interfaceto choose the location of the screen where they remember the targetbeing presented. After the user clicks or touches to indicate alocation, the actual location is indicated with a target displayed forabout several seconds on the screen, and the distance between theindicated and target position is recorded. Next, the subsequent image ispresented and the user again selects the remembered target location. Theorder of image presentation may be the same as during training or theorder of the images may be shuffled. A testing session may be repeatedone or more times. The number of times the testing session is repeatedmay be fixed (e.g. three repeated testing sessions) or the number ofrepeated testing sessions may be chosen according to a threshold ofminimum performance. Due to the feedback about the correct targetlocation provided after each image presentation, subjects generallyimprove their performance over multiple testing sessions.

The accuracy with which users select a remembered target location can becompared between different memory enhancement or memory disruptionconditions to determine the extent of memory modulation, if any.

Using an embodiment of a device for enhancing memory similar to thatshown in the schematic of FIGS. 11-19, healthy adults were tested in avisuospatial memory task as just described, and similar to that shown inFIGS. 20 and 21. The results are summarized in FIG. 22. The memory taskfor training was a visuospatial ‘concentration’-style game in which thesubject learns the location of pairs of identical images.

During training, a scent was released by an atomizer controlled by amicrocontroller. The subject practiced the memory game in the evening,and a baseline memory score was determined. The subject slept wearingthe sleep monitor (forehead electrodes) of the system for monitoringbrain rhythms. When SWS was detected, the portable, user-actuated systemincluding the controller (microcontroller), which was placed at thesubject's bedside, activated a scent atomizer in the experimentalcondition. In the placebo condition (‘vehicle’, FIG. 22), the scent wasreleased during training mode when the subject was practicing the gamebut not during sleep. In the morning, subjects were tested on the memorygame practiced the preceding night.

Pilot data were collected using software and hardware recording theresults of the trial in subjects using the device, and in subjects forwhom the memory consolidation step was not performed. This preliminarydata (n=3 per condition) showed successful enhancement of memory.Subjects recalled a larger proportion of image locations in the morningunder the experimental conditions relative to the placebo conditions(FIG. 22).

To save stimuli resources and help the user relate highly connectedinformation, when possible the system or user may choose to use anidentical stimulus or similar stimuli to train multiple, highly linkedpieces of learning content. For example, a quack could be played toremember facts both about the geography of the United States andPresidents of the United States.

In some variations, a system for modulating memory consolidation may bemore effective if it includes presenting effective contextual sensorystimuli during wakefulness when memory encoding occurs (e.g., duringstudying or experience of events that are desirable to be remembered) aswell as presenting related sensory stimuli during sleep when memoryconsolidation occurs. A computerized “learning database” is an effectivesystem for selecting, prioritizing, and scheduling which stimuli topresent at a particular time of wakefulness or sleep for a particularuser. In some variations, the learning database is stored onmachine-readable memory contained on a system for modulating memoryconsolidation wearably attached or near a user. In an embodiment, thelearning database is a SQLite or other database stored on a user'smobile handset (e.g. iPhone, iPod touch, Android phone) or tabletcomputer (e.g. iPad or Android tablet). In an embodiment, the learningdatabase is stored remotely on a computer or server accessible throughthe Internet (e.g., Amazon S3 or Dropbox). Any database structure thatenables storing, querying, and delivering information related to thememory consolidation process can be used. In some embodiments, customdatabase structures are used for a learning database.

Effective selection of contextual sensory stimuli for a specifictraining session may support modulation of memory consolidation duringsleep. An important feature for such a system is a learning databaseconfigured to store information about what sensory stimuli have beenpresented to or experienced by a user. In some embodiments, the learningdatabase stores data that was or can be used to generate a sensorystimulus as would occur for a database configured to store audio files(e.g. mp3 files) that can be played through appropriate computer andspeaker systems. In other embodiments, the learning database isconfigured as a relational database and stores a reference, ID number,link, or other symbolic reference to a particular sensory stimulus. Forinstance, the descriptive tag ‘rose smell’ can be stored in thedatabase. In some variations, a processed version of a contextualstimulus is stored in the learning database. Dimensional reductionanalysis techniques (e.g. principal components analysis, adaptivedimensional reduction through supervised learning, spectral analysiswithin key frequency bands) can be applied to sound files for instanceto identify dominant features in a particular sound, song, or audiorecording. Sounds of birds chirping or waves crashing may be distinctfrom each other yet share common features as defined on the basis of thedistribution power in various acoustic frequencies or other temporalfeatures of the auditory stimulus.

In some variations, an appropriately configured learning database isused to select contextual sensory stimuli that are sufficiently distinctfrom contextual sensory stimuli previously presented to the user by asystem for modulating memory consolidation. In some embodiments, acomputerized algorithm compares database entries about previouscontextual sensory stimuli presented to a user with a new set ofpotential contextual sensory stimuli, then selects one or more newstimuli to present. In some embodiments, several stimuli are selectedand they are prioritized to determine the order they will be used ascontextual sensory stimuli during user's study session.

In some variations, the algorithm is configured to select contextualsensory stimuli that are perceptually or statistically distinct frompreviously presented stimuli. Actual stimuli and/or pointers topreviously presented stimuli may be stored in the learning database. Inembodiments with a learning database configured to store a reference,pointer, or other symbolic representation of a contextual stimuli(rather than a machine-readable version of the stimulus itself), thesystem further comprises a system to access a stimulus from thereference, pointer, or symbolic representation—or access additionalinformation (such as descriptive hashtags, i.e. ‘waves crashing’,‘sound’, ‘environmental’)—in order to classify the sensory stimulus.

In some variations, an algorithm for comparing potential new stimuli topreviously presented stimuli is configured to have strict criteria forselecting contextual sensory stimuli such that a new sensory stimulus tobe presented cannot be similar to previously presented stimuli. Forinstance, if a subject has previously been presented with a sound ofwaves crashing on a beach, a subsequent similar stimulus, such as thesound of a waterfall, could be excluded. In other variations, apotential new sensory stimulus that is similar to a previously presentedsensory stimulus is presented with low priority (i.e., only if neededdue to extensive study sessions by the user such that all higherpriority contextual stimuli have already been presented).

In some variations, an algorithm for comparing potential new stimuli topreviously presented stimuli is configured to have a loose criteria,such that a similar but distinct stimulus is chosen to be presented to auser. In this embodiment, a recording of waves crashing on a beach couldbe chosen so long as it was not identical to a previous recording ofwaves crashing on a beach presented to the user during a study sessionand/or sleep session. In some variations, a statistical likelihood of asensory stimulus being perceptually distinct from another stimulus isdetermined based on an algorithm (e.g. weighting of dominant principalcomponents) and used as a criteria for selecting a sensory stimulus.

In some variations, similar sensory stimuli are intentionally chosenwhen similar content is presented in a study session. The system can beconfigured to select sensory stimuli from a similar modality and/or withsimilar perceptual, conceptual, or statistical features. The system isconfigured to determine what type of study material is being studied(and in some cases presented by the system) and automatically match aparticular type of sensory stimulus to it. In one exemplary variation,environmental sounds (e.g. birds chirping, waves crashing) areassociated with study of math concepts; while musical electronic(non-environmental) sounds are associated with engineering concepts.Geometry concepts similar to math would then be associated with asimilar environmental sound such as the sound of wind through leaves ona tree. A higher degree of specificity between stimuli and conceptswould also be beneficial in some contexts, such as when symphonicrecordings dominated by string instruments are associated with algebraconcepts and jazz recordings featuring saxophone solos are associatedwith study of geometry concepts.

In some variations, the set of potential contextual sensory stimuli arethe same sensory modality (e.g. all sounds, all smells, all tactilestimuli), while in other embodiments the set of potential contextualsensory stimuli include stimuli to activate different sensorymodalities. In another variation, hybrid sensory stimuli that includestimuli for activating more than one sensory modality are included (e.g.a sound and a smell presented together or with a fixed temporalinterval).

Systems and Methods for Selecting Distinct Contextual Sensory Stimulifor One or More Specific Training Sessions

The hardware and software of a learning database can be configured inseveral ways for generating, selecting, prioritizing, and/or schedulingcontextual sensory stimuli to be presented during one Study Mode period.The learning database is further configured to store the identity (or areference to the identity) of any contextual sensory stimulus presentedduring a Study mode period, so that it can be scheduled forre-representation during sleep. As described in greater detail below, insome embodiments, the learning database is configured so that somecontextual sensory are de-prioritized for re-presentation during sleepand accordingly may not actually be re-presented during sleep.

In some variations, a learning database system is configured so thatcontextual sensory stimuli are pre-selected before a training sessionbegins based on the content being studied or based on features of anexperience to be remembered. Pre-selected stimuli can be universallyassociated with particular study content that is commonly studied by apopulation of users (such as SAT vocabulary words or human anatomy termsrequired for passing a medical school exam). In this manner, thepre-selected contextual stimuli are not scheduled to be presented at aparticular time or in a particular order. Rather, the learning databaseis configured to have an association between a set of learning contentand one/or more contextual sensory cues. When a user studies thatparticular content with a memory enhancement system, the learningdatabase is queried, the appropriate sensory cue or set of cues isidentified, and the learning database transmits the appropriate cue tobe presented. One variation of this form of the invention would be aclip of a Bach symphony being associated with the names of the bones ofthe hand. In some embodiments, the learning database pre-selects a setof cues and associates them with a particular user or set of users inthe system, then determines the quality of memory enhancement for theassociated learning content and cue identity for each of the cues. Overtime, as more data is acquired concerning the effectiveness ofparticular cues for enhancing memories related to particular content,the system can use artificial intelligence or other algorithms tooptimize cue-content association and update the learning databaseaccordingly. In an embodiment, one component of the learning database isa quantitative or qualitative metric of confidence in the effectivenessof a cue-content pair. The confidence metric is then updated as new datais acquired, incorporated in the learning database, and used as an inputto an appropriate statistical algorithm.

In some variations a learning database system is configured so thatcontextual sensory stimuli are pre-selected before a training sessionbegins for a particular user. Features of contextual sensory stimuliselected or prioritized for association as a sensory-tag cue includesensory modality, intensity or salience, length (e.g. how long is anauditory clip; some individuals may have longer attention for a cuewhile other users may require cues to change more quickly as a functionof content being studied), and other features of the contextual sensorycue. For embodiments that use auditory cues, features of auditory clipscan be used to prioritize or select cues, including environmental sounds(e.g. birds chirping or highway traffic), songs (symphonic, clips fromlive music shows, riffs from a particular instrument, etc.), or abstractelectronic sounds. In an embodiment, the learning database is configuredto pre-select sensory cues based on a user's demographics, interests,etc. (age, sex, interests, friends' interests, any disabilities orsensory issues (color blindness, high frequency hearing loss, etc.)).

In some variations a learning database system is configured so thatcontextual sensory stimuli are pre-selected before a training sessionbegins based on a user's learning environment. The learning environmentcan be detected automatically by determining where the user is located.This can be accomplished by GPS components of a tablet computer or phonehandset (e.g. iPhone), by proximity sensors (e.g. RF) associated with aparticular location, or by the user manually indicating their locationvia a user interface on a static computerized system (e.g. one fixed inplace at the study environment, as in a library) or via anothermessaging system (e.g. text message, tweet, entry into a dedicated webpage, etc.). In this variation, the learning database is configured tohave an association between a location, environment, or behavioralcontext for the user (e.g., as a student in a classroom) and one or morecontextual sensory stimuli. For instance, smells are good in a privatestudying environment, as well as in a shared learning environment, butthe issues with air flow and controlling intensity of stimulus can be aproblem if a user enters Study Mode while outside or in a large room(e.g. a large lecture hall). Similarly, the learning database canautomatically prioritize sounds as a contextual cue—and even particulartypes of sounds—if a user is in a public location and using headphones.In some embodiments, statistical algorithms are used to make a ‘bestguess’ about a user's location or environment.

In some variations a learning database system is configured so thatcontextual sensory stimuli are pre-selected based on the time of day,day of the week, or time of year. In some variations the learningdatabase is configured to select particular cues at a particular time ofday. In an exemplar variation, calmer sounds are chosen during eveningstudy times while more energetic sounds are selected in the earlymorning or after lunch when a user may benefit from cues that areenergizing. In some variations the learning database is configured toselect particular cues for a particular day of the week. Similar to howthe system assigns, selects, or prioritizes sounds based on the time ofday, users may benefit from having cues that are more energizing onMondays and require a different set of cues on the weekend. In somevariations the learning database is configured to select particular cuesfor a particular time of the year. In an exemplar variation, ‘holiday’themed cues (i.e. songs related to Christmas, Hanukah, etc. or smellsassociated with the holiday season such as egg nog) may be effective. Inother embodiments, cues are pre-selected to provide contrast with commonenvironmental stimuli occurring at that time of year (e.g., winterholiday themed cues in July). In this exemplary embodiment, the learningdatabase may be configured not to select cues present in the user'senvironment, so that during the winter holidays, cues would be selectedto have little perceptual or statistical relationship with sounds,smells, or tastes generally associated with this time of year.

In some variations a learning database system is configured to selectcontext sensory stimuli in a pre-determined order. In an exemplarembodiment, 10 sounds or 5 smells are pre-selected for a user's firststudy session, and a new set of sounds and smells will be presented onsubsequent study sessions. In some variations a user orders a ‘pack’ ofnew sounds through a web interface or ‘in app’ purchase, then thesepurchased cues are presented in a fixed order. Users can selectindividual cues that they like (or sets of cues), similar to howindividuals can purchase ringtones for their phone.

In some variations a learning database system is configured to selectcontext sensory stimuli algorithmically. In some variations, thelearning database is configured to have software for achieving a desiredsensory cue-selection algorithm. In some variations the selection of asensory cue occurs approximately in real-time (right before the stimulusneeds to be delivered). In some variations the next stimulus or stimulito be presented to a user can be queued up, ready for when they begin astudy mode. In some variations multiple sets of stimuli can be queuedup, and the selection of one or more of the stimuli is contingent uponother factors such as the time of day, location of the user, content tobe studied, etc. In an advantageous embodiment, software algorithms toprocess or algorithmically select stimuli (i.e. to personalize themaccording to frequency component or the amount of time they are rampedon and off) can occur ‘in the cloud’ at a convenient time before a userenters a study mode (i.e. on an Amazon EC2 server according to a bidthat runs it when server time is cheap such as in the middle of thenight). In another embodiment of the invention, the learning database isconfigured to select a cue randomly, stochastically, or statistically.

In some variations a learning database system is configured so that athird party can select one or more contextual sensory stimuli. Invarious embodiments, the third party is a teacher, coach, friend,company (i.e. for marketing), or other entity, group, or individual. Theinvention may be further configured so that the third party can make thecontextual sensory cue selection by communicating with the learningdatabase. Communication with the learning database can occur via textmessage, tweet, API call, Internet form, voice-to-text, or other mode ofcommunication. In some variations a third party is a company and the cueis or is related to a jingle or otherwise supports marketing efforts ofthe company and thus represents a sponsorship opportunity. In somevariations a third party is an artist who selects specific cues toassociate with their art (or which are part of their art, as for soundclips used by an electronic musician) so as to trigger a specific set ofmemories related to an artistic performance. In this way the experienceof the artist's art has the potential for a further layer of control bythe artist—not just what the audience experiences, but also what theyretain over time. In an embodiment, the selection of one or morecontextual sensory stimuli is ‘crowdsourced’. A group of individualsselects a stimulus on behalf of the user.

In some variations the learning database generates a set of desiredcontextual sensory stimuli to present. In some cases, the members ofthis set may be prioritized concerning what sensory stimuli to presentin which order. The learning database and memory enhancement system isfurther configured to assess the sensory stimulation hardware andrelated components available to present stimuli to a user at therequested time and determines which of the desired set of contextualsensory stimuli can be presented. In the case wherein some stimulicannot be presented, the system will skip or deprioritize them forpresentation at another time. These circumstances may occur for instanceif a user has a speaker or headphones available to present an audiostimulus but no odorizers or similar hardware for presenting anolfactory stimulus. Accordingly, all olfactory stimuli that have beenprioritized or scheduled for presentation will be skipped or delayeduntil another time when appropriate smell generating hardware isavailable. This embodiment of the invention would also be useful if auser has low fidelity headphones incapable of presenting a low (bass)tone effectively, so sounds that have dominant low frequency componentswould be skipped or delayed until another time when appropriate soundgeneration hardware is available.

In some embodiments of the invention, the contextual sensory cuedelivered during sleep mode is a modified version of the cue presentedduring an earlier study mode period. In some variations the originalstimulus is modified to improve the efficiency with which stimuli aredelivered in one or more of the following ways: a stimulus that istemporally varying (e.g. a sound or spatiotemporal pattern of tactilestimulation) is sped up; one or more short clips are presented from astimulus that is extended in time (e.g. a 10 minute environmentalrecording or song); a stimulus is masked or otherwise modifying astimulus so that it is less likely to wake the user or anotherindividual sleeping near the user (e.g. on a background of white noise);a stimulus is ramped up and down in intensity when it begins and ends;multiple stimuli delivered during an earlier study session are presentedconcurrently in a combined stimulus so that multiple memory traces areactivated concurrently. A variation wherein two or more stimuli arecombined may also be advantageous for tying related concepts together inmemory. For instance, learning related equations about particle physicswith two different sound cues, then tying these two equations togetherduring sleep mode by playing the cues concurrently, overlapping in time,or subsequent to each other. In some instances, a desirable associationbetween two related concepts is not emphasized during the initiallearning of the material. For example, a student may be learning anexponential equation in a mathematics lecture and learning the growthrates of bacteria in a biology lab, but only emphasizes the associationduring sleep through presentation of combined stimuli. In oneembodiment, sound cues are presented during sleep mode on an unobtrusivebackground that is minimally distracting such as white noise or anotherrelaxing auditory stimulus that is unlikely to wake a user or othernearby sleeper.

Scheduling Re-Play of Stimuli

Described herein are systems and methods for selecting stimuli that havebeen previously presented during training for presentation again duringsleep and scheduling of memory reactivation events across nights ofsleep. Long-term use of a memory enhancement system based onsensory-tags presented to a user during study and re-presentation ofthese tags during sleep leads to a number of unexpected system designchallenges. First, sleep is limited in time. Particular stages of sleepthat may be conducive to memory consolidation (and enhancement of thisprocess) are further limited. For instance, slow-wave sleep occursduring a minority of a sleep period—and is particularly limited in someindividuals, including those with some sleep disorders, certaincognitive disorders (e.g. Alzheimer's disease and Down syndrome), and asa consequence of normal aging due to the known gradual decrease in theproportion of time spent in slow-wave sleep as individuals age. Due tothe limited time spent in slow-wave sleep and other stages of sleep,systems for scheduling which stimuli to re-present are advantageous.

Optimal memory formation may require intermittent reinforcement. Thisfeature of learning and studying is well known to psychologists,cognitive scientists, and neuroscientists but has not been previouslyconsidered in the context of modulation of memory consolidation duringsleep. Thus, the learning database and scheduler system is configured ina particularly beneficial embodiment of the invention to intermittentlyreinforce memory consolidation by re-presenting contextual sensorystimuli presented during a (much) earlier study session. In anembodiment of the invention, a contextual sensory stimulus (‘stimulusA’) is associated with learned content during a particular study periodby the user. In an embodiment of the invention, the learning databaseand scheduler system are configured to present ‘stimulus A’ duringslow-wave sleep (or another phase of sleep) at a time chosen from thegroup of: a nap during the same day when the study session occurredduring which ‘stimulus A’ was presented; a nap on a subsequent day afterthe study session occurred during which ‘stimulus A’ was presented; thefirst night of sleep on the evening immediately following the studysession occurred during which ‘stimulus A’ was presented; or on a nighta variable number of nights after the ‘stimulus A’ study session, wherethe amount of time between the study session and the firstre-presentation of ‘stimulus A’ is chosen from the list consisting of: 2nights, 3 nights, 4 nights, 5 nights, 6 nights, 7 nights, 8 nights, 9nights, 10 nights, 2 weeks, 3 weeks, 4 weeks, 5 weeks, 6 weeks, 7 weeks,8 weeks, 3 months, 6 months, 1 year or longer.

In some variations the learning database and scheduler system areconfigured to re-present one or more contextual sensory stimuli duringsleep that were previously presented to the user during a study modeperiod. Re-presenting the same stimulus to trigger memory consolidation,memory re-consolidation, or disruption of memory consolidation orre-consolidation is a beneficial feature of the invention becausesystems configured in this manner (and methods for achieving thisembodiment of the invention) enable more reliable or stronger memoriesto be stored by improving memory consolidation but do not requireadditional study sessions during wakefulness.

There are often large amounts of information to learn, consolidate, andretain, yet limited periods of sleep.

Everyday situations often require a large amount of information to beretained by the user. In such cases, the user may find himself/herselfscheduling an overwhelming number of reinforcing stimuli, despite thelimited amount of available slow-wave sleep for sleeping reinforcement.In an unmanaged situation, the desired sleep-reinforcing event may notoccur within the desired time window for learning the information. Thus,systems and methods for determining how to schedule contextual sensorystimuli presented during a study session to be presented during sleepwould be beneficial for making a broadly useful memory enhancementsystem that can be used over days, weeks, months, and years.

In some embodiments, the system tracks the number of awake learningevents undertaken for each piece of information (when the user used amemory enhancement system in ‘Study Mode’). In an embodiment, usersassign priority to each piece of information learned or experiencedthrough a ranking, scoring, or other prioritization system. In anembodiment, the user prioritizes information acquired using a tablet,mobile phone, web interface, or other software system. One system thathas similar functionality for ranking and organizing information in thismanner through a web browser or mobile device is Trello(www.trello.com). A component of the system described here is ascheduler to determine which contextual sensory stimuli to re-presentduring a particular sleep epoch. The schedule is optionally adjustedbased on the rankings and prioritization given by the user. Inalternative embodiments, prioritization of content for memoryenhancement in a user is done by a third party individual, company, orservice. For all learning content that had been previously associatedwith a contextual cue, the learning database is configured to store anassociation between the content and cue. In this manner, a user can rankthe content for priority of learning without needing to be aware of thecues that had been associated during study with content.

Based on the prioritization system chosen by or on behalf of the user,the learning database is queried for the associated contextual sensorycue presented and that cue is presented to the sleeping user during anappropriate stage of sleep according to the order defined by theprioritized learning list. In some circumstances, de-prioritizingmemories that do not need to undergo enhanced memory consolidationduring sleep is beneficial. De-prioritization of content can be done fora variety of reasons, including: if the user already has formed a strongmemory (has learned) the related content; or if the content is lessimportant than other content for the user to retain (e.g. for material ateacher has told the student will not be tested).

In some variations the system is configured to alert the user if anystimuli have not been re-presented during sleep over a period of time.This situation would occur for instance if there is a backlog of sensorycues previously associated with learned content that has been classifiedas lower priority. In various embodiments, the period of time is chosenfrom the group consisting of: less than one day, more than 1 day, morethan 2 days, more than 3 days, more than 4 days, more than 5 days, morethan 6 days, more than 7 days, more than 2 weeks, more than 1 month, orlonger. In some embodiments, the user creates an ‘alert’ or calendarentry for particular learned content. The alert or calendar entry wouldbe used to schedule re-presentation of the contextual sensory cue tohappen before a given date, on a given date, or after a given data. Auser may want to have re-presentation occur before a test or quiz andwould thus create a calendar entry the day or some fixed number of daysbefore the quiz (but the user would be agnostic as to when during theperiod preceding the calendar entry the re-presentation event occurs).Alternatively, a user may want to have re-presentation occur on a givendate, and the system would schedule that item to be a high priority cuefor re-presentation during a sleep epoch on that day (or the nightfollowing that day). In optional embodiments of the invention, thesystem is configured to automatically generate an alert to the user ifthere are too many scheduled cues for re-presentation in a given period.In an alternative embodiment, the system is configured to suggest abedtime or amount of sleep required in order to fulfill the full numberof priority re-presentation events. In an embodiment, the systemsuggests to the user that they take a nap due to the large number ofsensory re-presentation events scheduled. The alert generated by thesystem can optionally be sent by one or more methods chosen from thegroup of: email, phone call, voice message, text message, direct messagetweet, or another form of communication. In another embodiment, apassive alert to sleep is presented to the user in the form ofsuggestive sensory stimuli. For example, a user that performs a fixedroutine just prior to sleep, including using fragrant soap, may bealerted by the system to go to sleep by delivering a similar fragrantodor. In short, this is an alternate framework for considering what itmeans to have ‘sleep debt’. The term ‘sleep debt’ is generallyassociated with the concept of amount or quality of sleep, but in thiscontext is associated with not having sufficient sleep time forre-presenting contextual sensory stimuli.

Presenting a contextual sensory stimulus more than once during one ormore epochs of slow-wave sleep (or another phase of sleep) or during oneor more nights of sleep can be advantageous for further strengtheningthe memory consolidation process for the related learned content. Thenumber of repeated presentations of a sensory stimulus is chosen fromthe group of more than once, more than twice, more than three times,more than four times, more than five times, more than 10 times, morethan 15 times, more than 25 times, more than 50 times, more than 100times, more than 500 times, more than 1000 times, more than 10000 times,or more frequently. In an embodiment, different modified versions of acontextual sensory stimulus or clips from a stimulus that is extended intime are presented. In this way, the triggering of particular learningcontent is repeated but the cue itself is slightly different and thusless susceptible to habituation by the appropriate sensory system orsystems. In some embodiments, a contextual sensory stimulus associatedwith lower priority learned content is re-presented fewer times duringone or more epochs of slow-wave sleep than it would otherwise have been.

In an embodiment, the learning database and/or scheduler is configuredto automatically assign priority of learned content according topersonalized criteria that can include demographic and other information(e.g. age, sex, IQ, academic strengths and weaknesses, interests,assignments (from work, school, etc) and other information).

As sleep is limited in time, some embodiments optimize the number ofpresentations delivered to a user.

Closed-loop feedback may be used. For example, in some variations theuser is able to notify the scheduler system as to whether the desiredmemory has been successfully modulated at any time after the initiallearning session for that memory. The scheduler subsequently alters thedelivery schedule of the tagging ‘stimulus A’. In the particular case ofmemory enhancement, the scheduler may perform either of: ceasingdelivery of the ‘stimulus A’ (allowing other tagging stimuli to bepresented during SWS), or increasing the frequency of planned instanceswhen ‘stimulus A’ is to be delivered for passive maintenance of amemory. Alternatively, if the user reports failed memory enhancement,the scheduler can decrease the frequency of planned instances of‘stimulus A’ delivery.

Closed-loop feedback—individualized scheduling—may also or alternativelybe used. In some variations user reporting of successful andunsuccessful memory consolidation can be used by the scheduler tooptimize the timing of stimulus delivery. Consistent user reporting ofoutcome allows the scheduler in desired instances to create astatistical model of outcome success as a function of variousstimulation parameters, such as time of stimulus presentation, timeintervals between stimulus presentation, as well as user physiologicalparameters, such as sleep quality, duration of sleep or nap, and heartrate. As a result of this statistical analysis, the scheduler can adjustthe scheduled delivery of stimuli in a manner optimal for the user,given his prior performance history. In some embodiments, reporting ofoutcome can be done automatically such as via a self-testing program ona website or the reporting of outcome can be done by a third party suchas a teacher or tutor.

Systems and Methods for Scheduling Sleep

An advantageous optional feature of the learning database and sensorycue selection and scheduling system may include the ability to‘schedule’ sleep for a user. In some variations the system interfaceswith a user's calendar via a Google calendar API, Microsoft Outlook, apersonal assistant who manages the user's memory enhancement, or othercalendaring system and ‘schedule’ a nap or longer (or shorter) night ofsleep for a user in order to achieve particular memory enhancementeffect or with the intention that the user will enter a particular phaseof sleep for a certain amount of time.

One or more of the following factors, measurements, or user demographicscan be used to further adjust the scheduler of sleep: (1) a user'sstudying and memory enhancement needs—during an intensive period ofstudy (e.g., a recent law school graduate spending the summer studyingfor the bar exam) or during a period of time when re-activation ofpreviously consolidated memories would be advantageous (e.g., for astudent in the days or week before a final exam, such that previouslyenhanced memories from earlier in the semester can be furtherconsolidated); (2) a user's recent or historical sleep, slow-wave sleep,or another sleep state as self-reported or measured by a sleep sensorcomponent of a memory enhancement system or by another sleep measurementsystem or device; (3) a user's physiology, including one or more ofmeasurement of brain rhythms, heart rate (and heart rate variability),galvanic skin response, breathing rate, pupil dilation, and otherphysiological measurements that can be used to infer or determine auser's cognitive, physiological, emotional, or attentional state withrelation to memory encoding, consolidation, and retrieval; (4) a user'sdemographic information such as age, sex, health, weight, or othercomorbidities that affect sleep or memory processes or both (sleepapnea, a sleep disorder, mild cognitive impairment, normal aging,Alzheimer's disease, other forms of dementia, stroke, chemotherapytreatment, radiation therapy, Down syndrome, attention-deficithyperactivity disorder, etc); and (5) a third-party's needs for memoryenhancement by the user, such as a teacher, tutor, corporate trainer, ormilitary/first responder training or debriefing protocol.

Exemplary systems and methods for scheduling repetition of contextualsensory stimulus presentation during sleep are described herein. In somevariations the learning database and scheduler system is configured topresent a contextual sensory stimulus is presented multiple times in asingle epoch of sleep. The number of times a stimulus is presentedduring a single night of sleep or epoch of a sleep stage during a nightof sleep is generally chosen from the list of at least once, at leasttwice, at least three times, at least 4 times, at least 5 times, atleast 10 times, at least 15 times, at least 25 times, at least 100times, or more times, depending on the length of the night of sleep orsleep epoch and the length of time of a single sensory stimulus. In anembodiment, the learning database and scheduler system is configured topresent a contextual sensory stimulus during multiple sleep epochs in asingle night. Generally, there are more than one and less than 10 epochsof slow-wave sleep in a single night.

In some variations the learning database and scheduler system isconfigured to present multiple contextual sensory stimuli sequentiallyin a single epoch of slow-wave sleep or another sleep stage. Thescheduler system can be optionally configured to deliver or trigger thedelivery of sensory stimuli in an order chosen according to one or moreof the following frameworks: (1) sensory stimuli presented according tothe priority of the learned content associated with them duringstudying; (2) in the order in which the sensory stimuli were presentedduring studying; (3) in the reverse order which the sensory stimuli werepresented during studying; (4) in an order that assigns a lower priority(later presentation) for sensory stimuli that were presented at thebeginning of a study session due to the well-known ‘Primacy Effect’wherein memories are stronger for items at the beginning of a list orstudy session; (5) in an order that assigns a lower priority (laterpresentation) for sensory stimuli that were presented at the end of astudy session due to the well-known ‘Recency Effect’ wherein memoriesare stronger for items at the end of a list or study session; (6) in arandom order; (7) in an order structured so as to achieve repetitionsfor a particular sensory stimulus or set of sensory stimuli severaltimes in succession before moving on to a next sensory stimulus; or (8)ordering rules customizable by a user or third party via a userinterface.

In embodiments wherein a contextual sensory stimulus that extends intime (generally a sound, spatiotemporal tactile stimulus, or visualstimulus) is delivered to a subject during study, the scheduler isappropriately configured for re-presentation of all or part of that cueduring sleep. In an embodiment, the entire sensory cue is re-presentedduring slow-wave sleep or another sleep state. In some embodiments, theepoch of slow-wave sleep ends before the entire sensory cue has beenpresented, in which case the system can be configured to stop the replayof the sensory stimulus. In another embodiment, brief clips of thesensory stimulus are re-presented during sleep. The clips can be chosento be from: the beginning, the end; key transition points in the sensorycue (e.g. for a song, at the beginning of a verse or a time when asaxophone solo occurs in a live jazz recording—these times periods canmarked by a third party (e.g. hired via oDesk or Amazon MechanicalTurk), or the time periods can be detected algorithmically by analyzingthe waveform of the song); random periods in the stimulus; or clipswithin the stimulus that have been shown by other users to be effectiveas sensory-tags for enhancing memory consolidation. In an embodiment, invarious repetitions of the cue in a single night or across multiplenights can use different clips from the same sensory stimulus.

In some variations appropriate for use with both short and long sensorystimuli, processed versions of a stimulus are presented during the sleepphase. For instance, an audio file can be processed using dimensionalreduction analysis techniques to identify the most salient aspects ofthe stimulus. This embodiment can also employ signal processingtechniques for audio files that take advantage of what is known abouthuman hearing, such as by using known audio compression techniques (e.g.mp3). Principal component analysis and Fourier transforms of the audiofile could also be beneficial to use as stimuli.

In some embodiments, the scheduler is configured to adjust the salienceor intensity of a contextual sensory stimulus. In some embodiments,optimization of the salience of intensity of a contextual sensorystimulus is done during study. Factors including the stimulus, usersensory thresholds, background noise, etc. can affect the requiredsalience during both study modes and sleep modes. In an embodiment, thesalience or intensity of a tone is adjusted across multiplepresentations of a cue within one epoch of slow-wave sleep or acrossmultiple epochs of SWS. For instance, for sounds the sound pressurelevel can be changed with each iteration of a sensory stimulus duringstudy and/or sleep (e.g. the sound gets gradually louder until itreaches a predefined upper bound or the sound begins at silence or aminimum loudness, then ramps up in sound pressure level to a predefinedupper limit). For smells, the amount of odorant released can be adjustedto affect the salience or intensity of a contextual sensory cue.

In variations wherein a contextual sensory stimulus that extends in time(generally a sound, spatiotemporal tactile stimulus, or visual stimulus)is delivered to a subject during study, the scheduler is appropriatelyconfigured for re-presentation of all or part of that cue during sleep.In an embodiment, the entire sensory cue is re-presented duringslow-wave sleep or another sleep state. In some embodiments, the epochof slow-wave sleep ends before the entire sensory cue has beenpresented, in which case the system can be configured to stop the replayof the sensory stimulus. In another embodiment, brief clips of thesensory stimulus are re-presented during sleep. The clips can be chosento be from: the beginning, the end; key transition points in the sensorycue (e.g. for a song, at the beginning of a verse or a time when asaxophone solo occurs in a live jazz recording—these times periods canmarked by a third party (e.g. hired via oDesk or Amazon MechanicalTurk), or the time periods can be detected algorithmically by analyzingthe waveform of the song); random periods in the stimulus; or clipswithin the stimulus that have been shown by other users to be effectiveas sensory-tags for enhancing memory consolidation. In some variationsvarious repetitions of the cue in a single night or across multiplenights can use different clips from the same sensory stimulus.

In variations appropriate for use with both short and long sensorystimuli, processed versions of a stimulus may be presented during thesleep phase. For instance, an audio file can be processed usingdimensional reduction analysis techniques to identify the most salientaspects of the stimulus. This embodiment can also employ signalprocessing techniques for audio files that take advantage of what isknown about human hearing, such as by using known audio compressiontechniques (e.g. mp3). Principal component analysis and Fouriertransforms of the audio file could also be beneficial to use as stimuli.

Systems and Method for Use with Electronic Learning Media

As mentioned above, the apparatus, systems and methods described hereinmay be used with any appropriate source of learning or training,including electronic book readers and other electronic media, such asweb pages/browsers. A system for improving memory having a training modeand a sleep consolidation mode may be configured to be used for enhancedlearning of electronic media, where the form of electronic media isselected from the group including, but not limited to: electronic books(‘e-books’ or ‘e-readers’); text content served via the Internet to acomputing device; audio recordings played by a computerized system;video recordings played by a computerized system; classroom lectures;dynamic or interactive software systems (i.e. games); electronic braillereaders; text-to-speech audio; and other forms of multimedia includingtext, images, audio, and video.

In particular, the invention can be used to modulate how electronicmedia material is learned by enhancing or disrupting memories associatedwith all or part of the electronic media material. In an advantageousembodiment, the electronic media can be material accessed on a website,mobile/tablet application, or e-reader.

In contrast to traditional books, electronic media accessed through theInternet (or via an Internet-enabled device) may be particularlyadvantageous, because the memory improvement system can be configured torecord information to a computer memory or remote server about thecontent experienced, as well as the time, location, presented sensorystimuli, and other meta-data about the training session. These data areuseful for providing feedback to the user or for reviewing previouslyexperienced material.

The system may be configured so that the experienced electronic mediarepresents a training session. In order to form an association betweenthe content learned during a training session, the system enterstraining mode and presents one or more sensory stimuli concurrently witha user's experience (e.g., reading, watching, listening) of selectedelectronic media. The presented sensory stimuli are then presented tothe user when the user is in a specified sleep state.

Sensory stimuli can be of any modality. Beneficial sensory modalitiescan be chosen depending on the form of the electronic media beingexperienced. For instance, auditory cues may be presented eithercontinuously or intermittently while the user experiences textual and/orvisual electronic media. Auditory cues may also be advantageous forelectronic media that takes the form of audio recordings or videorecordings, but an auditory cue needs to be selected to be one that isnondisruptive to hearing the audio portions of the experiencedelectronic media.

Some modalities of sensory stimuli presented during experience ofelectronic media require additional hardware. Olfactory stimuli can bedelivered through an odorizer, volatile odor, candle, or other system.Tactile stimuli can be presented through a wearable array of buzzers orother tactile transducers. In some embodiments that require additionalhardware for creating sensory stimuli, the hardware is manuallycontrolled by the user. In other embodiments that require additionalhardware for creating sensory stimuli, the hardware is configured to becontrolled by a computer, e-reader, mobile device, smartphone, or othercomputerized system via a wired or wireless protocol to define when theodor is released and related parameters.

The system may register the sensory stimuli that are presented during atraining session and schedules them to be presented again to thesleeping user during a specified sleep stage during a period of sleepthat follows the training session, as mentioned above.

In some variations, the system is configured to enhance learning ofcontent experienced in e-books, including textbooks, works ofnon-fiction, works of fiction, and volumes focused on visual experiencesuch as graphic novels or books about design, art, or photography.

In some variations of the invention an e-book is experienced by a userby a computerized system chosen from the list including, but not limitedto: dedicated e-reader such as an Amazon Kindle or Barnes and NobleNook; an app running on a tablet, mobile device, or smartphone (e.g.Kindle app on an iPhone); a web-interface on a desktop computer, laptop,tablet, mobile device, or smartphone; or standalone software foraggregating and/or managing content.

A useful feature of the invention may be that sensory stimuli,experienced content, and metadata may be uploaded to a company serverand available for feedback to the user, improvement of user experience,sharing via social media, and data mining.

Another useful optional feature of the system is automatic recognitionthat a user has experienced an instance of electronic media during aprevious training session. In short, a user is reviewing material. Thus,the system may detect “review” versus novel learning. In this case, thesame sensory cue may be automatically presented, if possible givensensory transducers available at that time—e.g. if the user hadpreviously experienced an olfactory cue and an appropriate olfactorytransducer is available when material is reviewed, then the same orsimilar olfactory cue is presented; however if an appropriate olfactorycue cannot be generated during the review of previously experiencedmaterial and an alternative transducer is available (e.g., for sound),then a new stimulus is presented. In this way, a stronger associationcan be formed by re-presenting a same or largely similar sensorystimulus.

Thus, for example, any of the apparatus and systems described herein maybe configured to integrate training mode functionality with third-partyelectronic media content experienced via the Internet on a website, amobile or tablet ‘app’, software programs running on a desktop or laptopcomputer, or standalone hardware. The may be beneficial because itenables a user to enhance or disrupt memories of their choosingexperienced through third party software and/or hardware. In variousadvantageous embodiments, a plug-in, API, bookmarklet, widget, softwarepackage, software library, or other software architecture is used tointegrate training mode functionality with third-party electronic media.

This embodiment may also add an important portability and flexibility toa memory modulation system by permitting its integration with contentexperienced via a web browser, content aggregator, or other appropriatesoftware (e.g., Evernote).

Optionally, integration can occur by having an app, plug-in, or othersoftware structure running as a background process when a user accessescontent on a third-party site. Optionally, the app runs as a backgroundprocess on an e-book, mobile device, tablet, smartphone, or otherelectronic system.

The following examples describe illustrative examples of the invention.These examples are not exhaustive, and may include any additionalfeatures or elements described herein. In addition, additional oralternatively configurations and methods of operation are possible, andmay be reasonably understood from the disclosure herein.

Example 6 Electronic Media

Sensory cues can be pre-defined for particular electronic media andautomatically presented to the user as they experience the chosencontent during a training session. Auditory, visual, and tactile cuesare easily controlled in the temporal domain and thus advantageousmodalities for being pre-defined for a particular electronic media ore-book.

Optionally, sensory cues are configured to occur intermittently as auser experiences an e-book, audio recording, video recording, or otherinstance of electronic media. By pre-defining the cues for particularcontent, important portions of content can be preferentially tagged bysensory stimuli. For instance, a new set of sensory stimuli is presentedfor a summary section at the end of each chapter of a textbook.Audio-linked e-books are available for instance through the company‘Booktrack’.

For audio recordings and video recordings, the playback rate is known,so sensory stimuli can be selected to match the duration of theelectronic media. Further, sensory stimuli can be selected orconstructed to have thematic elements occurring concurrently withsalient or informative portions of the electronic media so that thesesections can be prioritized for reactivation during sleep.

For text content, different users read at different speeds, meaning thatsensory stimuli cannot be perfectly aligned with the content. In anembodiment when a user reads text (and/or views images), sensory stimuliare constructed or selected to be sufficiently long such that it doesnot matter how long a user takes to get through a section. If they moveto a new section of content, then the system skips to a next track,keeping a record of which portions of the cue were presented. The systemonly replays the portion of sensory cue they actually experienced duringa training session as a sleep sound.

Optionally, the system can be configured to be adaptive, for instance byapplying a machine learning algorithm to estimate the speed with whichthey get through material of a particular type, difficulty, or area ofexpertise—and use the estimate of reading speed to select sensorystimuli of appropriate duration or structure.

Optionally, the system can be configured by a user by selecting anestimated reading speed (i.e. in words per minute). The reading speed isused by the system to estimate how long a user will require to getthrough a section, defining the duration or structure of sensorystimulus selected.

Optionally, eye tracking can be used to determine a user's progress inreading text. Alternatively, a user can indicate their position withintext by a touch screen interaction, by moving a mouse, pressing a mousebutton, by a keyboard interface, or by remote control. Speed readingapps present one/few word(s) at a time, so the stimulus coordination appcan know exactly what the user is currently learning. For exampleReadQuick: http://mashable.com/2012/08/31/readquick-review/

Optionally, if a sensory stimulus finishes before a user completes asection, a second sensory stimulus is selected and presented.

Optionally, a repeating sensory feature such as a melodic theme orrhythm for an auditory stimulus can be presented whenever a particularconcept or topic occurs in the experienced multimedia material.

This feature may be advantageous because it partially circumvents thelimited period when a specified sleep stage occurs (e.g. slow-wavesleep, which occurs from 10 s of minutes per night to a few hours pernight). The limited amount of a specified sleep stage limits the timewhen sensory stimuli can be presented during sleep and limits how manymemories can be processed during sleep. By only replaying the thematicelement of a sensory stimulus, more content can be processed duringsleep due to associated with a smaller number of cues.

Optionally, unique sensory stimuli are presented when a user reads aspecific page, paragraph, or set of words. For example, as a desiredcontent page is approached the sound of a wrinkle may be played.

Example 7 Electronic Playlist

In one variation, a user chooses her own playlist of auditory stimuli(i.e. sounds). In another variation, a user selects one or more genresof music or artists that they enjoy to be the basis of which auditorycues are presented. Any software for playing audio files on electronicmedia can be used, including streaming web audio (e.g., Rdio, Spotify,Pandora, or SoundCloud), audio stored locally on a user's hardware (e.g.iTunes), audio played from physical media (e.g. played via a flashdrive, compact disc, or record player), or personal music stored in thecloud (e.g. iCloud or Amazon Cloud Player).

Optionally, a user can enter their login credentials for a third partysite to allow the memory modulation system to access their music libraryor other data.

Optionally, the system can keep track for a user what content has beenpresented during a previous training session so that a unique cue isavailable for a new training session.

Optionally, the system is configured to make an API call to skip to anext audio recording (e.g. ‘track’) or select a new audio recording.

In some cases, when a user selects her own playlist, experiencedelectronic media material cannot be aligned to the content and/orstructure of the learned material during a training session because theaudio files are not matched to the length of time needed to experiencethe learned material. Optionally, a user can manually move on to a newtrack for a new section of material. In this case, the system isconfigured to store the portion of the audio file that was presented sothat non-presented portions of the song are not replayed to the userduring sleep.

Example 8 Multimedia Learning Experience

If learning experience on e-book or other platform involves video withsound or audio, then (if using auditory or visual cues) the cues coulddistract from the experience of the content. In some variations, thesystem is configured to automatically recognize the format of theelectronic media and deliver audio stimuli designed to be nondisruptiveto hearing the sounds of the audio and/or video content.

Ambient music and structured white noise are examples of types ofauditory cues that would be minimally disruptive to experiencedelectronic media that incorporates an auditory component.

For instance, a video recording of a lecture requires that a studenthear the lecturer's voice. The system may recognize that the learningcontent has an auditory component based on the file type, metadata, orother identifier of the electronic media, then selects an ambient musicstimulus to play at a background sound level that does not distract theuser from hearing the lecturer's voice on the video recording.

Thus, in any of the variations described above, the system may be usedin conjunction with electronic media; further, the system may tailor thesensory stimuli used based on the content of the electronic media. Thus,the system may pre-review or analyze the electronic media in order todetermine the sensory stimuli to apply. For example, the system (e.g.,media analysis module of the system) may analyze the electronic media todetermine one or more of: (1) the type of sensory stimuli (e.g., sensemodality and/or particular sensory stimuli within a sensory modality)based on the content of the electronic media; (2) the duration of thesensory stimuli based on the pacing/duration of the electronic media;(3) the volume of the sensory stimuli (based on overlap with other mediawithin the sensory modality of the sensory stimuli, etc.

From the foregoing it will be seen that this invention is one welladapted to attain all ends and objects hereinabove set forth togetherwith the other advantages which are obvious and which are inherent tothe structure. It will be understood that certain features andsubcombinations are of utility and may be employed without reference toother features and subcombinations. This is contemplated by and iswithin the scope of the claims. Since many possible embodiments may bemade without departing from the scope thereof, it is to be understoodthat all matter herein set forth or shown in the accompanying drawingsis to be interpreted as illustrative, and not in a limiting sense.

What is claimed is:
 1. A system for improving learning, memory, orlearning and memory from an electronic media, the system having atraining mode and a sleep consolidation mode, the system comprising: asleep monitor configured to monitor a user's sleep state; a sensorystimulator configured to provide a plurality of distinct sensory stimulibased on the electronic media; a controller comprising control logicconfigured to select a sensory stimulus for a specific training sessionbased on the content of the electronic media, and to control theapplication of the sensory stimulus, wherein the controller receivesinformation on the user's sleep state from the sleep monitor, andcontrols the sensory stimulator to apply the contextual sensory stimulusfrom the specific training session according to a training schedule whenthe user is experiencing a specified sleep stage during a sleepconsolidation mode; and a learning database configured to be populatedwith training content and sensory stimuli co-presented with the trainingcontent, wherein the controller is configured to determine which sensorystimuli to use based on the electronic media.
 2. The system of claim 1,wherein the sensory stimulator is configured to provide an ambientsensory stimulus recorded during the training session.
 3. The system ofclaim 1, wherein the controller is configured to determine which sensorystimuli to use based on the content of the electronic media.
 4. Thesystem of claim 1, wherein the controller is configured to determine howto apply the sensory stimuli based on the content of the electronicmedia.
 5. The system of claim 1, wherein the controller is configured todetermine which sensory stimuli to use based on the structure of theelectronic media.
 6. The system of claim 1, further comprising a memoryconfigured to store information that indicates one or more of whichsensory stimuli have been applied for specific training sessions, thesleep state of user, and completion of application of a sensory stimulusduring a sleep consolidation mode following a specific training session.7. The system of claim 1, wherein the sensory stimulator is configuredto deliver one or more of: olfactory stimuli, auditory stimuli andtactile stimuli.
 8. The system of claim 1, wherein the electronic mediacomprises an electronic book.
 9. The system of claim 1, furthercomprising a communications module coupled to the controller configuredto allow communication with a remote site.
 10. The system of claim 1,wherein the control logic comprises an application configured to beexecuted on a mobile device.
 11. The system of claim 1, furthercomprising an analysis module configured to pre-analyze the electronicmedia to determine a sensory stimuli to apply.
 12. A portableuser-controllable device for improving learning and memory from anelectronic media, the device comprising: a user interface comprising acontrol allowing a user to place the device into a training mode beforethe user experiences the electronic media; a sensory stimulatorconfigured to present a plurality of distinct sensory stimuliconcurrently with a user's experience of the electronic media; sleepmonitoring logic configured to determine when the user is in a specifiedsleep state; and a controller comprising control logic configured todetermine a sensory stimulus based on the electronic media for aparticular training session; wherein the controller is furtherconfigured to reapply the specific sensory stimulus when the user is inthe specified sleep state following the training session; wherein thecontroller is configured to determine which sensory stimuli to use basedon the content, structure or content and structure of the electronicmedia.
 13. The device of claim 12, wherein the sleep monitoring logic isconfigured to determine when the user is in a specific sleep state usinguser motion and heart rate.
 14. The device of claim 12, wherein thecontrol logic comprises a set of distinct sensory stimuli to bepresented during a particular training session.
 15. The device of claim12, wherein the sensory stimulator is configured to deliver one or moreof olfactory stimuli, auditory stimuli, and tactile stimuli.